Re: [Caml-list] Providing module implementations for Oasis

[Ivan Gotovchits <ivg@ieee.org>]

[-- Attachment #1: Type: text/plain, Size: 9762 bytes --]

The problem is that executable `ppx_forest` and library `forest_ppx` both
refer to the same compilation
unit `ppx_forest`.  So the solution is easy, just remove the `Ppx_forest`
module from the library (indeed, it is not part of the library, it is the
executable implementation).
Here is the PR with the fix: https://github.com/padsproj/oforest/pull/9.

# Details

Just in case if you're interested why the resulting error was so confusing
:)

The error message came from the following compilation command:

    ocamlfind ocamlopt -linkpkg -g -linkpkg -thread -package threads
-package str -package re.str -package re.glob -package re -package
ppx_tools.metaquot -package ppx_let -package ppx_deriving.show -package
pads.ppx -package pads -package core -package compiler-libs.common
ppx/forest_ppx.cmxa parsing/forest_parser.cmxa lib/forest.cmxa
ppx/ppx_forest.cmx -o ppx/ppx_forest.native


It is obvious that the command is incorrect (pun intended). Let's,
actually, remove all the packages so that we can see the libraries:

    ocamlfind ocamlopt -linkpkg ppx/forest_ppx.cmxa
parsing/forest_parser.cmxa lib/forest.cmxa ppx/ppx_forest.cmx -o
ppx/ppx_forest.native

In fact, the problem is in library ordering. Modules and libraries must be
sorted in topological order, so the first should be `lib/forest.cmxa`, then
`parsing/forest_parser.cmxa` and finally `forest_ppx.cmxa`.
And, yes, this is how this command looks after the fix:

    ocamlfind ocamlopt -linkpkg lib/forest.cmxa parsing/forest_parser.cmxa
ppx/forest_ppx.cmxa ppx/ppx_forest.cmx -o ppx/ppx_forest.native

So how the toposort can be broken? My assumption, that this was because you
introduced a loop into a dependency graph, when you added ppx_forest to
both: the library and the executable.
If my assumption is true, then probably we should add to ocamlbuild
toposort routine an easy check that will detect loops and output  a proper
diagnostic message if a dependency graph contains cycles.

Regards,
Ivan Gotovchits


On Fri, Oct 28, 2016 at 2:14 AM, Jonathan DiLorenzo <
dilorenzo@cs.cornell.edu> wrote:

> Hmm, sadly this seems to still produce the same error after cleaning up
> all the oasis generated files and everything:
>
> File "_none_", line 1:
> Error: No implementations provided for the following modules:
>          Forest_parser_helper referenced from ppx/forest_ppx.cmxa(Skins)
> Command exited with code 2.
>
> Maybe notable (or maybe obvious) that if I switch oasis to use ocamlc
> instead of ocamlopt, I instead get:
>
> File "_none_", line 1:
> Error: Error while linking ppx/forest_ppx.cma(Skins):
> Reference to undefined global `Forest_parser_helper'
> Command exited with code 2.
>
> New _oasis file (not with best -> byte change):
>
> Library forest_parser
>   Path:           parsing
>   BuildTools:     ocamlbuild, menhir, ocamllex
>   Findlibparent:  forest
>   Findlibname:    forest_parser
>   BuildDepends:   forest, ppx_deriving.show, compiler-libs.common
>   CompiledObject: best
>   Modules:  Forest_parser_helper, Forest_types
>   InternalModules: Forest_lexer, Forest_parser
>
> Library forest_ppx
>   Path:           ppx
>   BuildTools:     ocamlbuild
>   Findlibparent:  forest
>   Findlibname:    forest_ppx
>   BuildDepends:   re, re.str, forest.forest_parser, forest,
> ppx_tools.metaquot
>   CompiledObject: best
>   Modules:        Ppx_forest
>   InternalModules: Ppx_forest_lib, Utility, Skins
>   XMETAEnable:    true
>   XMETADescription: Syntax extension library for OCaml Forest
>   XMETARequires:   str re core threads ppx_tools.metaquot
>   XMETAExtraLines:  ppx = "ppx_forest"
>
> Executable ppx_forest
>   Path:           ppx
>   MainIs:         ppx_forest.ml
>   BuildDepends:   forest, forest.forest_parser, forest.forest_ppx,
> pads.ppx, ppx_tools.metaquot
>   CompiledObject: best
>
>
> On Thu, Oct 27, 2016 at 2:24 PM, Ivan Gotovchits <ivg@ieee.org> wrote:
>
>> The problem is  in a name clashing between `$opam-switch/lib/ocaml/compiler-libs/parser`
>> and the `parser` library that is compiled from your internal library named
>> `parser.cmxa`.
>> The compiler-lib is added to the search path by the `ppx_tools` library,
>> and when you're trying to link your final executable, it takes the wrong
>> archive, that definitely doesn't have
>> `Forest_parser_helper` module.
>>
>> The reason, why it is still able to see that there is no `bad_func` in
>> `Forest_parser_helper` is because the interfaces are not contained in the
>> `cmxa` file, but are looked up directly in
>> the `cmi`. Compiler is looking for a file named
>> `forest_parser_helper.cmi` and can see that there are not `bad_func` there.
>>
>> The solution is to rename your library, e.g., use `Library forest_parser`
>> instead of `Library parser`
>>
>> On Thu, Oct 27, 2016 at 1:44 PM, Jonathan DiLorenzo <
>> dilorenzo@cs.cornell.edu> wrote:
>>
>>> Hey,
>>>
>>> Ok, now the next step would be the following. Carefully check that all
>>>> modules, that you're using in libraries are included in `Modules` or
>>>> `InternalModules`. It is the case
>>>> that sometimes when you forgot to include a module, oasis (actually
>>>> `ocamlbuild`), may produce a strange, and on a first glance, irrelevant
>>>> error message.
>>>>
>>>
>>> I checked and sadly the modules I'm trying to access are all in
>>> `Modules`. Perhaps interestingly, it can still clearly detect statically if
>>> the function exists or not because if I try to use a function that doesn't
>>> exist it instead gives me this error (which is what I would normally
>>> expect):
>>>
>>> File "ppx/skins.ml", line 39, characters 8-37:
>>> Error: Unbound value Forest_parser_helper.bad_func
>>> Command exited with code 2.
>>>
>>>
>>>> If it doesn't help then the `_build/_log` file might help us to debug
>>>> the issue.
>>>>
>>>
>>> Attached. Thanks so much for helping with this again. I'm at a total
>>> loss.
>>>
>>>
>>>>
>>>> On Thu, Oct 27, 2016 at 10:59 AM, Jonathan DiLorenzo <
>>>> dilorenzo@cs.cornell.edu> wrote:
>>>>
>>>>> Hey Ivan,
>>>>>
>>>>> Thanks for your response.
>>>>>
>>>>> The first one is that you didn't run `oasis setup` after you made the
>>>>>> changes, so I would suggest cleaning current state
>>>>>> and starting from scratch. (I usually just do `git clean -idx` just
>>>>>> to be sure that I got rid of any generated files, like setup.data, et alas,
>>>>>> but be careful,
>>>>>> don't delete something that you need).
>>>>>>
>>>>>
>>>>> I tried cleaning up all the generated files. Same error unfortunately
>>>>>
>>>>>
>>>>>> Another idea is that you're using oasis 0.4.7 with
>>>>>> ocaml-4.03.0+flambda, they are currently incompatible, and can produce
>>>>>> weird bugs. If that so, then consider switching
>>>>>> either a compiler or oasis to different versions.
>>>>>>
>>>>>
>>>>> I am using oasis 0.4.7 and ocaml-4.03.0, but I don't appear to be
>>>>> using flambda. I tried passing in -config and it said that flambda was
>>>>> false at least, but I admit, I'm not especially familiar with it, so if
>>>>> there's some other way I should be checking for this please let me know.
>>>>> Hopefully, they're compatible sans flambda?
>>>>>
>>>>>
>>>>> On Thu, Oct 27, 2016 at 1:01 AM, Jonathan DiLorenzo <
>>>>>> dilorenzo@cs.cornell.edu> wrote:
>>>>>>
>>>>>>> Hey all,
>>>>>>>
>>>>>>> I'm trying to build my system using Oasis and OCamlbuild. I have 3
>>>>>>> different libraries and for some reason I seem to only be able to refer
>>>>>>> from one to the other in seemingly random files. For example, I want my
>>>>>>> 'ppx' library to be able to use functions from my 'parser' library. One
>>>>>>> file (ppx_forest.ml) can use functions from it, while another in
>>>>>>> the same library (skins.ml) cannot, giving me this error:
>>>>>>>
>>>>>>> File "_none_", line 1:
>>>>>>> Error: No implementations provided for the following modules:
>>>>>>>          Forest_parser_helper referenced from ppx/ppx.cmxa(Skins)
>>>>>>> Command exited with code 2.
>>>>>>>
>>>>>>> They can all refer to types I've defined therein though.
>>>>>>>
>>>>>>> I'm not fantastic at Oasis so maybe I'm just missing something
>>>>>>> basic? Any ideas? Any more information I can gather to give more context
>>>>>>> otherwise?
>>>>>>>
>>>>>>> Thanks in advance,
>>>>>>> Jonathan
>>>>>>>
>>>>>>> Excerpt of my Oasis file that may possibly be relevant:
>>>>>>>
>>>>>>> Library forest
>>>>>>>   Path:          lib
>>>>>>>   BuildTools:    ocamlbuild
>>>>>>>   BuildDepends:   pads, str, re, re.glob, core, threads, ppx_let
>>>>>>>   CompiledObject: best
>>>>>>>   Modules:       Forest, PadsInterface
>>>>>>>
>>>>>>> Library parser
>>>>>>>   Path:           parsing
>>>>>>>   BuildTools:     ocamlbuild, menhir, ocamllex
>>>>>>>   Findlibparent:  forest
>>>>>>>   Findlibname:    parser
>>>>>>>   BuildDepends:   forest, ppx_deriving.show, compiler-libs.common
>>>>>>>   CompiledObject: best
>>>>>>>   Modules:  Forest_parser_helper
>>>>>>>   InternalModules: Forest_lexer, Forest_parser, Forest_types
>>>>>>>
>>>>>>> Library ppx
>>>>>>>   Path:           ppx
>>>>>>>   BuildTools:     ocamlbuild
>>>>>>>   Findlibparent:  forest
>>>>>>>   Findlibname:    ppx
>>>>>>>   BuildDepends:   re, re.str, forest.parser, forest,
>>>>>>> ppx_tools.metaquot
>>>>>>>   CompiledObject: best
>>>>>>>   Modules:        Ppx_forest
>>>>>>>   InternalModules: Ppx_forest_lib, Utility, Skins
>>>>>>>   XMETAEnable:    true
>>>>>>>   XMETARequires:   str re core threads ppx_tools.metaquot
>>>>>>>   XMETAExtraLines:  ppx = "ppx_forest"
>>>>>>>
>>>>>>> Executable ppx_forest
>>>>>>>   Path:           ppx
>>>>>>>   MainIs:         ppx_forest.ml
>>>>>>>   BuildDepends:   forest, forest.parser, forest.ppx, pads.ppx,
>>>>>>> ppx_tools.metaquot
>>>>>>>   CompiledObject: best
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

[-- Attachment #2: Type: text/html, Size: 15288 bytes --]