[Caml-list] oasis building out-of-src and camlp4

[Caml-list] oasis building out-of-src and camlp4

[Magnus Therning]

[-- Attachment #1: Type: text/plain, Size: 2622 bytes --]

I've been trying to get oasis to build completely out-of-source.
Basically I want to use it as an ExternalProject in a C/C++ project that
uses CMake.

If I have the following files

~~~ oasis-simple/_oasis
OASISFormat : 0.4
Name : oasis-simple
Version : 0.0
Synopsis : A simple test of building oasis
Authors : Magnus Therning
License : BSD-3-clause

Executable "oasis-simple"
    Path : src
    MainIs : main.ml
    BuildTools : ocamlbuild
    BuildDepends : oasis
~~~

~~~ oasis-simple/src/main.ml
let () = print_string "Hello, World!"
~~~

then I can build out-of-source using the following steps:

    % mkdir ../oasis-simple-build; cd ../oasis-simple-build
    % oasis setup -oasis ../oasis-simple/_oasis
    % ocaml setup.ml -C $PWD/../oasis-simple -configure
    % ocaml setup.ml -C $PWD/../oasis-simple -build -build-dir $PWD/build

If I make it a little more complicated, by adding a syntax extension:

~~~ oasis-camlp4/_oasis
OASISFormat : 0.4
Name : oasis-camlp4
Version : 0.0
Synopsis : A slightly more complicated test of building oasis
Authors : Magnus Therning
License : BSD-3-clause

Executable "oasis-camlp4"
    Path : src
    MainIs : main.ml
    BuildTools : ocamlbuild
    BuildDepends : oasis, core, sexplib.syntax, threads
~~~

~~~ oasis-camlp4/src/main.ml
open Core.Std

type colour = Red | Green | Blue
    with sexp

let () = printf "Hello, World! (%s)\n" (sexp_of_colour Green |> Sexp.to_string)
~~~

Then I follow the same formula as above:

    % mkdir ../oasis-camlp4-build; cd ../oasis-camlp4-build
    % oasis setup -oasis ../oasis-camlp4/_oasis
    % ocaml setup.ml -C $PWD/../oasis-camlp4 -configure
    % ocaml setup.ml -C $PWD/../oasis-camlp4 -build -build-dir $PWD/build

Which results in an error:

~~~
+ /usr/bin/ocamldep.opt -modules src/main.ml > src/main.ml.depends
File "src/main.ml", line 12, characters 4-8:
Error: Syntax error
Command exited with code 2.
Compilation unsuccessful after building 1 target (0 cached) in 00:00:00.
E: Failure("Command ''/usr/bin/ocamlbuild' src/main.byte -tag debug -build-dir /home/magnus/devo/tmp/oasis-camlp4-build/build' terminated with error code 10")
~~~

What am I missing here, how do I get the second example to build?

/M

-- 
Magnus Therning                      OpenPGP: 0xAB4DFBA4 
email: magnus@therning.org   jabber: magnus@therning.org
twitter: magthe               http://therning.org/magnus

There does not now, nor will there ever, exist a programming language
in which it is the least bit hard to write bad programs.
     -- Flon's Axiom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [Caml-list] oasis building out-of-src and camlp4

[Anil Madhavapeddy]

On 22 Sep 2015, at 23:12, Magnus Therning <magnus@therning.org> wrote:
> 
> ~~~
> + /usr/bin/ocamldep.opt -modules src/main.ml > src/main.ml.depends
> File "src/main.ml", line 12, characters 4-8:
> Error: Syntax error
> Command exited with code 2.
> Compilation unsuccessful after building 1 target (0 cached) in 00:00:00.
> E: Failure("Command ''/usr/bin/ocamlbuild' src/main.byte -tag debug -build-dir /home/magnus/devo/tmp/oasis-camlp4-build/build' terminated with error code 10")
> ~~~
> 
> What am I missing here, how do I get the second example to build?

The ocamldep invocation here is not passing the -pp parameter to run
the source code through camlp4.  I'm not sure how you fix that in
CMake, but add the relevant lines (which should be the same as those
passed to ocamlc/ocamlopt) to ocamldep and it should get further...

regards,
Anil

Re: [Caml-list] oasis building out-of-src and camlp4

[Magnus Therning]

[-- Attachment #1: Type: text/plain, Size: 1460 bytes --]

On Wed, Sep 23, 2015 at 09:48:43AM +0200, Anil Madhavapeddy wrote:
> On 22 Sep 2015, at 23:12, Magnus Therning <magnus@therning.org> wrote:
> > 
> > ~~~
> > + /usr/bin/ocamldep.opt -modules src/main.ml > src/main.ml.depends
> > File "src/main.ml", line 12, characters 4-8:
> > Error: Syntax error
> > Command exited with code 2.
> > Compilation unsuccessful after building 1 target (0 cached) in 00:00:00.
> > E: Failure("Command ''/usr/bin/ocamlbuild' src/main.byte -tag debug -build-dir /home/magnus/devo/tmp/oasis-camlp4-build/build' terminated with error code 10")
> > ~~~
> > 
> > What am I missing here, how do I get the second example to build?
> 
> The ocamldep invocation here is not passing the -pp parameter to run
> the source code through camlp4.  I'm not sure how you fix that in
> CMake, but add the relevant lines (which should be the same as those
> passed to ocamlc/ocamlopt) to ocamldep and it should get further...

Ah, OK, I'll have to look into that.

I should point out though that I've not started using CMake yet.  The
examples I showed use just `oasis` (with `ocamlbuild` as its tool).  I'm
still trying to work out the incantations to use, the integration with
CMake comes later :)

/M

-- 
Magnus Therning                      OpenPGP: 0xAB4DFBA4 
email: magnus@therning.org   jabber: magnus@therning.org
twitter: magthe               http://therning.org/magnus

LeBlanc's law: Later equals never.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [Caml-list] oasis building out-of-src and camlp4

[Gabriel Scherer]

Thanks for the reproducible information. I just tried and could
reproduce the issue.

The problem comes from the fact that, with the command-line invocation
you use oasis generates the build description files (_tags,
myocamlbuild.ml, setup.data) in the build directory, not in the source
directory, but ocamlbuild when invoked (from the source directory)
will look for them in the source directory. In particular, I could
"fix" the build by doing

  ...
  ocaml setup.ml -C $PWD/../oasis-camlp4 -configure
  for f in {myocamlbuild.ml,_tags,setup.data}; do ln -s $PWD/$f
../oasis-camlp4; done
  ocaml setup.ml -C $PWD/../oasis-camlp4 -build -build-dir $PWD/build

It looks like oasis's behavior with "-C $DIR -configure" is
problematic: it should generate its files in $DIR -- except maybe
setup.ml, or then people would have to run (ocaml $DIR/setup.ml -C
$DIR ...).

TL;DR: this appear to be an oasis issue, rather than a issue with
CMake or ocamlbuild.

On Wed, Sep 23, 2015 at 10:19 AM, Magnus Therning <magnus@therning.org> wrote:
> On Wed, Sep 23, 2015 at 09:48:43AM +0200, Anil Madhavapeddy wrote:
>> On 22 Sep 2015, at 23:12, Magnus Therning <magnus@therning.org> wrote:
>> >
>> > ~~~
>> > + /usr/bin/ocamldep.opt -modules src/main.ml > src/main.ml.depends
>> > File "src/main.ml", line 12, characters 4-8:
>> > Error: Syntax error
>> > Command exited with code 2.
>> > Compilation unsuccessful after building 1 target (0 cached) in 00:00:00.
>> > E: Failure("Command ''/usr/bin/ocamlbuild' src/main.byte -tag debug -build-dir /home/magnus/devo/tmp/oasis-camlp4-build/build' terminated with error code 10")
>> > ~~~
>> >
>> > What am I missing here, how do I get the second example to build?
>>
>> The ocamldep invocation here is not passing the -pp parameter to run
>> the source code through camlp4.  I'm not sure how you fix that in
>> CMake, but add the relevant lines (which should be the same as those
>> passed to ocamlc/ocamlopt) to ocamldep and it should get further...
>
> Ah, OK, I'll have to look into that.
>
> I should point out though that I've not started using CMake yet.  The
> examples I showed use just `oasis` (with `ocamlbuild` as its tool).  I'm
> still trying to work out the incantations to use, the integration with
> CMake comes later :)
>
> /M
>
> --
> Magnus Therning                      OpenPGP: 0xAB4DFBA4
> email: magnus@therning.org   jabber: magnus@therning.org
> twitter: magthe               http://therning.org/magnus
>
> LeBlanc's law: Later equals never.

Re: [Caml-list] oasis building out-of-src and camlp4

[Magnus Therning]

[-- Attachment #1: Type: text/plain, Size: 1873 bytes --]

On Wed, Sep 23, 2015 at 10:38:31AM +0200, Gabriel Scherer wrote:
> Thanks for the reproducible information. I just tried and could
> reproduce the issue.
> 
> The problem comes from the fact that, with the command-line invocation
> you use oasis generates the build description files (_tags,
> myocamlbuild.ml, setup.data) in the build directory, not in the source
> directory, but ocamlbuild when invoked (from the source directory)
> will look for them in the source directory. In particular, I could
> "fix" the build by doing

Indeed, the goal is to not contaminate the source directory *at all*.

>   ...
>   ocaml setup.ml -C $PWD/../oasis-camlp4 -configure
>   for f in {myocamlbuild.ml,_tags,setup.data}; do ln -s $PWD/$f
> ../oasis-camlp4; done
>   ocaml setup.ml -C $PWD/../oasis-camlp4 -build -build-dir $PWD/build
> 
> It looks like oasis's behavior with "-C $DIR -configure" is
> problematic: it should generate its files in $DIR -- except maybe
> setup.ml, or then people would have to run (ocaml $DIR/setup.ml -C
> $DIR ...).

I've been playing around a bit with the various combinations of where I
run `ocaml <path>/setup.ml` and `-C`. Since the steps work on "simple"
code, but doesn't on code using `camlp4` I'm leaning towards calling
this a bug.

> TL;DR: this appear to be an oasis issue, rather than a issue with
> CMake or ocamlbuild.

Yes, I'm guessing it's an issue in the code generated by `oasis` and how
it uses `ocamlbuild` and other build tools.

/M

-- 
Magnus Therning                      OpenPGP: 0xAB4DFBA4 
email: magnus@therning.org   jabber: magnus@therning.org
twitter: magthe               http://therning.org/magnus

The man who is denied the opportunity of taking decisions of
importance begins to regard as important the decisions he is allowed
to take.
     -- C Northcote Parkinson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [Caml-list] oasis building out-of-src and camlp4

[Gabriel Scherer]

> Indeed, the goal is to not contaminate the source directory *at all*.

You could, in the "external build" command passed to CMake, wrap the
build invocation with code to create the symlinks before the build,
and remove them after the build.

That said, If I understand oasis' philosophy correctly (but I'm far
from an expert), the file that are generated by the oasis setup are
not to be understood as "generated files never to be looked at", but
rather *source* files that you may want to modify and customize
yourself, and just contain some snippets produced by oasis. I would
understand this as a kind of "we created the basic boilerplate
structure of your project for you", rather than "those are hidden
files only used during the build". In particular, you may be expected
to include these files in your version-control repository (something I
have been a bit reluctant to do myself; I think that the newish
"dynamic" setup mode of oasis avoids versioning any cruft).

(That said, many users never have to touch the oasis-generated files.)

> Since the steps work on "simple"
> code, but doesn't on code using `camlp4` I'm leaning towards calling
> this a bug.

I'll let oasis developers triage this issue, but it is not as clear as
you describe. It's not that this oasis setup works in simple cases and
breaks in other, but rather than oasis does not support this setup at
all, yet sometimes ocamlbuild can do what it needs without any help
whatsoever.


One other option would be to enhance ocamlbuild to support the
scenario where the build-description files it needs (_tags and
myocamlbuild.ml) are located outside the source repository, which is a
reasonable assumption if we assume they have been generated rather
than user-written (your situation). We could add a (-config-dir $DIR)
option for that, or separate options (-plugin-source /path/to/foo.ml)
and (-tag-file /path/to/_tags). It strikes me as a bit odd to have a
_tags file in another directory, that contains glob patterns relative
to the source directory (in your case the _tags file will refer to
<src/...>). But I like the idea and created a feature request to track
it:
  http://caml.inria.fr/mantis/view.php?id=6997
But if you go this route, it means that you would have to wait for the
next ocamlbuild release, so an oasis-level solution or workaround
would currently be more helpful for you.

On Wed, Sep 23, 2015 at 10:54 AM, Magnus Therning <magnus@therning.org> wrote:
> On Wed, Sep 23, 2015 at 10:38:31AM +0200, Gabriel Scherer wrote:
>> Thanks for the reproducible information. I just tried and could
>> reproduce the issue.
>>
>> The problem comes from the fact that, with the command-line invocation
>> you use oasis generates the build description files (_tags,
>> myocamlbuild.ml, setup.data) in the build directory, not in the source
>> directory, but ocamlbuild when invoked (from the source directory)
>> will look for them in the source directory. In particular, I could
>> "fix" the build by doing
>
> Indeed, the goal is to not contaminate the source directory *at all*.
>
>>   ...
>>   ocaml setup.ml -C $PWD/../oasis-camlp4 -configure
>>   for f in {myocamlbuild.ml,_tags,setup.data}; do ln -s $PWD/$f
>> ../oasis-camlp4; done
>>   ocaml setup.ml -C $PWD/../oasis-camlp4 -build -build-dir $PWD/build
>>
>> It looks like oasis's behavior with "-C $DIR -configure" is
>> problematic: it should generate its files in $DIR -- except maybe
>> setup.ml, or then people would have to run (ocaml $DIR/setup.ml -C
>> $DIR ...).
>
> I've been playing around a bit with the various combinations of where I
> run `ocaml <path>/setup.ml` and `-C`. Since the steps work on "simple"
> code, but doesn't on code using `camlp4` I'm leaning towards calling
> this a bug.
>
>> TL;DR: this appear to be an oasis issue, rather than a issue with
>> CMake or ocamlbuild.
>
> Yes, I'm guessing it's an issue in the code generated by `oasis` and how
> it uses `ocamlbuild` and other build tools.
>
> /M
>
> --
> Magnus Therning                      OpenPGP: 0xAB4DFBA4
> email: magnus@therning.org   jabber: magnus@therning.org
> twitter: magthe               http://therning.org/magnus
>
> The man who is denied the opportunity of taking decisions of
> importance begins to regard as important the decisions he is allowed
> to take.
>      -- C Northcote Parkinson

Re: [Caml-list] oasis building out-of-src and camlp4

[Magnus Therning]

[-- Attachment #1: Type: text/plain, Size: 3057 bytes --]

On Wed, Sep 23, 2015 at 11:19:57AM +0200, Gabriel Scherer wrote:
> > Indeed, the goal is to not contaminate the source directory *at all*.
> 
> You could, in the "external build" command passed to CMake, wrap the
> build invocation with code to create the symlinks before the build,
> and remove them after the build.

Yes, CMake lets me do pretty much anything, but it does make the build a
little more complex and I'm already sneaking in OCaml without telling ;)

> That said, If I understand oasis' philosophy correctly (but I'm far
> from an expert), the file that are generated by the oasis setup are
> not to be understood as "generated files never to be looked at", but
> rather *source* files that you may want to modify and customize
> yourself, and just contain some snippets produced by oasis. I would
> understand this as a kind of "we created the basic boilerplate
> structure of your project for you", rather than "those are hidden
> files only used during the build". In particular, you may be expected
> to include these files in your version-control repository (something I
> have been a bit reluctant to do myself; I think that the newish
> "dynamic" setup mode of oasis avoids versioning any cruft).

Yes, I am aware of that, and with "dynamic" mode `oasis` seems to
generate little enough that I wouldn't mind putting it in VCS.

> > Since the steps work on "simple"
> > code, but doesn't on code using `camlp4` I'm leaning towards calling
> > this a bug.
> 
> I'll let oasis developers triage this issue, but it is not as clear as
> you describe. It's not that this oasis setup works in simple cases and
> breaks in other, but rather than oasis does not support this setup at
> all, yet sometimes ocamlbuild can do what it needs without any help
> whatsoever.

I tend to land in the camp of "if the behaviour is surprising (without
delighting) to a user, then it's a bug".  I'm fully aware that what I'm
trying to do might not have been a scenario the `oasis` devs ever
considered.

> <src/...>). But I like the idea and created a feature request to track
> it:
>   http://caml.inria.fr/mantis/view.php?id=6997

Thanks, I'll keep an eye on that request.

> But if you go this route, it means that you would have to wait for the
> next ocamlbuild release, so an oasis-level solution or workaround
> would currently be more helpful for you.

For the time being I get by with using `ocamlbuild` (via `corebuild`)
and its `-build-dir` option.  Hopefully there'll be more support for
out-of-source building when I get to the point of replacing all C code
with OCaml ;)

/M

-- 
Magnus Therning                      OpenPGP: 0xAB4DFBA4 
email: magnus@therning.org   jabber: magnus@therning.org
twitter: magthe               http://therning.org/magnus

You do not examine legislation in the light of the benefits it will
convey if properly administered, but in the light of the wrongs it
would do and the harms it would cause if improperly administered.
     -- Lyndon Johnson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [Caml-list] oasis building out-of-src and camlp4

[Sylvain Le Gall]

[-- Attachment #1: Type: text/plain, Size: 3675 bytes --]

Please consider adding more comments to the following bug, which seems
close to what you are discussing:
https://forge.ocamlcore.org/tracker/index.php?func=detail&aid=1473&group_id=54&atid=291

I must admit that I totally miss the whole point of the discussion,
probably if you can make a summary of what you precisely need in the BTS, I
will be able to understand the problem better.

Regards
Sylvain

Le mer. 23 sept. 2015 à 11:44, Magnus Therning <magnus@therning.org> a
écrit :

> On Wed, Sep 23, 2015 at 11:19:57AM +0200, Gabriel Scherer wrote:
> > > Indeed, the goal is to not contaminate the source directory *at all*.
> >
> > You could, in the "external build" command passed to CMake, wrap the
> > build invocation with code to create the symlinks before the build,
> > and remove them after the build.
>
> Yes, CMake lets me do pretty much anything, but it does make the build a
> little more complex and I'm already sneaking in OCaml without telling ;)
>
> > That said, If I understand oasis' philosophy correctly (but I'm far
> > from an expert), the file that are generated by the oasis setup are
> > not to be understood as "generated files never to be looked at", but
> > rather *source* files that you may want to modify and customize
> > yourself, and just contain some snippets produced by oasis. I would
> > understand this as a kind of "we created the basic boilerplate
> > structure of your project for you", rather than "those are hidden
> > files only used during the build". In particular, you may be expected
> > to include these files in your version-control repository (something I
> > have been a bit reluctant to do myself; I think that the newish
> > "dynamic" setup mode of oasis avoids versioning any cruft).
>
> Yes, I am aware of that, and with "dynamic" mode `oasis` seems to
> generate little enough that I wouldn't mind putting it in VCS.
>
> > > Since the steps work on "simple"
> > > code, but doesn't on code using `camlp4` I'm leaning towards calling
> > > this a bug.
> >
> > I'll let oasis developers triage this issue, but it is not as clear as
> > you describe. It's not that this oasis setup works in simple cases and
> > breaks in other, but rather than oasis does not support this setup at
> > all, yet sometimes ocamlbuild can do what it needs without any help
> > whatsoever.
>
> I tend to land in the camp of "if the behaviour is surprising (without
> delighting) to a user, then it's a bug".  I'm fully aware that what I'm
> trying to do might not have been a scenario the `oasis` devs ever
> considered.
>
> > <src/...>). But I like the idea and created a feature request to track
> > it:
> >   http://caml.inria.fr/mantis/view.php?id=6997
>
> Thanks, I'll keep an eye on that request.
>
> > But if you go this route, it means that you would have to wait for the
> > next ocamlbuild release, so an oasis-level solution or workaround
> > would currently be more helpful for you.
>
> For the time being I get by with using `ocamlbuild` (via `corebuild`)
> and its `-build-dir` option.  Hopefully there'll be more support for
> out-of-source building when I get to the point of replacing all C code
> with OCaml ;)
>
> /M
>
> --
> Magnus Therning                      OpenPGP: 0xAB4DFBA4
> email: magnus@therning.org   jabber: magnus@therning.org
> twitter: magthe               http://therning.org/magnus
>
> You do not examine legislation in the light of the benefits it will
> convey if properly administered, but in the light of the wrongs it
> would do and the harms it would cause if improperly administered.
>      -- Lyndon Johnson
>

[-- Attachment #2: Type: text/html, Size: 4795 bytes --]

Re: [Caml-list] oasis building out-of-src and camlp4

[Magnus Therning]

[-- Attachment #1: Type: text/plain, Size: 2233 bytes --]

On Tue, Sep 29, 2015 at 10:59:49PM +0000, Sylvain Le Gall wrote:
> Please consider adding more comments to the following bug, which seems
> close to what you are discussing:
>
> https://forge.ocamlcore.org/tracker/index.php?func=detail&aid=1473&group_id=54&atid=291
> 
> I must admit that I totally miss the whole point of the discussion,
> probably if you can make a summary of what you precisely need in the
> BTS, I will be able to understand the problem better.

Unfortunately I don't know enough about `oasis` to understand that bug
report :(

I'll try to explain how I'd like to use oasis/buildtool by looking how
Haskell's Cabal lets me do it:

- Configure without generating anything in the current dir:

    % runhaskell Setup.hs --builddir /tmp/my_build_dir

- Build using the just generated configuration, again without putting
  anything in the current dir:

    % runhaskell Setup.hs --builddir /tmp/my_build_dir

Comparing this to oasis/buildtool (assuming I've a pre-existing
`setup.ml` generated with `oasis setup -setup-update dynamic` to
minimize the amount of build-related stuff in the project):

- Configure

    % ocaml setup.ml -configure

  This generates a file, `setup.data` in the current dir.  I've found no
  way to have it generated in a dedicated build dir.

- Build

    % ocaml setup.ml -build -build-dir /tmp/my_build_dir -no-links

  This builds in the mentioned dir, but it also generates a `setup.log`
  in the current dir!

I thought that `-C` could offer a way to achieve this, but it doesn't:

    % mkdir /tmp/my_build_dir; cd /tmp/my_build_dir
    % ocaml /path/to/setup.ml -h
    Exception: Sys_error "_oasis: No such file or directory".
    File "/path/to/setup.ml", line 1:
    Error: Reference to undefined global `OASISDynRun'

I'm not convinced `-C` *should* be the way to achieve what I want.

Hopefully this makes it clearer what I mean.

/M

-- 
Magnus Therning                      OpenPGP: 0xAB4DFBA4 
email: magnus@therning.org   jabber: magnus@therning.org
twitter: magthe               http://therning.org/magnus

Any sufficiently advanced technology is indistinguishable from a rigged
demo.
     -- Andy Finkel

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [Caml-list] oasis building out-of-src and camlp4

[Sylvain Le Gall]

[-- Attachment #1: Type: text/plain, Size: 3728 bytes --]

Hello,

Le mer. 30 sept. 2015 à 13:49, Magnus Therning <magnus@therning.org> a
écrit :

> On Tue, Sep 29, 2015 at 10:59:49PM +0000, Sylvain Le Gall wrote:
> > Please consider adding more comments to the following bug, which seems
> > close to what you are discussing:
> >
> >
> https://forge.ocamlcore.org/tracker/index.php?func=detail&aid=1473&group_id=54&atid=291
> >
> > I must admit that I totally miss the whole point of the discussion,
> > probably if you can make a summary of what you precisely need in the
> > BTS, I will be able to understand the problem better.
>
> Unfortunately I don't know enough about `oasis` to understand that bug
> report :(
>
> I'll try to explain how I'd like to use oasis/buildtool by looking how
> Haskell's Cabal lets me do it:
>
> - Configure without generating anything in the current dir:
>
>     % runhaskell Setup.hs --builddir /tmp/my_build_dir
>
> - Build using the just generated configuration, again without putting
>   anything in the current dir:
>
>     % runhaskell Setup.hs --builddir /tmp/my_build_dir
>
> Comparing this to oasis/buildtool (assuming I've a pre-existing
> `setup.ml` generated with `oasis setup -setup-update dynamic` to
> minimize the amount of build-related stuff in the project):
>
> - Configure
>
>     % ocaml setup.ml -configure
>
>   This generates a file, `setup.data` in the current dir.  I've found no
>   way to have it generated in a dedicated build dir.
>
> - Build
>
>     % ocaml setup.ml -build -build-dir /tmp/my_build_dir -no-links
>
>   This builds in the mentioned dir, but it also generates a `setup.log`
>   in the current dir!
>
> I thought that `-C` could offer a way to achieve this, but it doesn't:
>
>     % mkdir /tmp/my_build_dir; cd /tmp/my_build_dir
>     % ocaml /path/to/setup.ml -h
>     Exception: Sys_error "_oasis: No such file or directory".
>     File "/path/to/setup.ml", line 1:
>     Error: Reference to undefined global `OASISDynRun'
>
> I'm not convinced `-C` *should* be the way to achieve what I want.
>
> Hopefully this makes it clearer what I mean.
>
>
First, why not just copy everything in the new directory:
% cp -R . /tmp/my_build_dir; cd /tmp/my_build_dir
% ocaml setup.ml -configure
% ocaml setup.ml -build

Maybe I miss some context to understand why this simple solution is not
enough.

Going back at the other points. There are 3 categories of files:
- the ones that are really generated at build time: setup.log, setup.data
  -> they probably need to follow the indication provided -C. This is a bug
- the ones that are generated by the underlying build system: _build
(ocamlbuild)
  -> as you wrote "-build -build-dir /tmp/my_build_dir -no-links" should be
enough
- the ones that are generated by the dynamic mode, which is a hack to
prevent adding the files that other tools need:
  -> these ones are tricky because they should have been in the source
directory right from the beginning and being able to apply -C to them is
equivalent to make sure that "ocamlfind"/"ocamlbuild"/other build tools
will use -C and the source directory... This is clearly not trivial to
assume that all tools will do.

My initial proposal to "cp" the files and build from the copy would
probably solve your problem without assuming that all tools comply with -C.
Tell me if it solves your problem.

Regards
Sylvain

/M
>
> --
> Magnus Therning                      OpenPGP: 0xAB4DFBA4
> email: magnus@therning.org   jabber: magnus@therning.org
> twitter: magthe               http://therning.org/magnus
>
> Any sufficiently advanced technology is indistinguishable from a rigged
> demo.
>      -- Andy Finkel
>

[-- Attachment #2: Type: text/html, Size: 5388 bytes --]

Re: [Caml-list] oasis building out-of-src and camlp4

[Magnus Therning]

[-- Attachment #1: Type: text/plain, Size: 4581 bytes --]

On Thu, Oct 01, 2015 at 06:46:55AM +0000, Sylvain Le Gall wrote:
> Hello,
> 
> Le mer. 30 sept. 2015 à 13:49, Magnus Therning <magnus@therning.org> a
> écrit :
> 
> > On Tue, Sep 29, 2015 at 10:59:49PM +0000, Sylvain Le Gall wrote:
> > > Please consider adding more comments to the following bug, which seems
> > > close to what you are discussing:
> > >
> > >
> > https://forge.ocamlcore.org/tracker/index.php?func=detail&aid=1473&group_id=54&atid=291
> > >
> > > I must admit that I totally miss the whole point of the discussion,
> > > probably if you can make a summary of what you precisely need in the
> > > BTS, I will be able to understand the problem better.
> >
> > Unfortunately I don't know enough about `oasis` to understand that bug
> > report :(
> >
> > I'll try to explain how I'd like to use oasis/buildtool by looking how
> > Haskell's Cabal lets me do it:
> >
> > - Configure without generating anything in the current dir:
> >
> >     % runhaskell Setup.hs --builddir /tmp/my_build_dir
> >
> > - Build using the just generated configuration, again without putting
> >   anything in the current dir:
> >
> >     % runhaskell Setup.hs --builddir /tmp/my_build_dir
> >
> > Comparing this to oasis/buildtool (assuming I've a pre-existing
> > `setup.ml` generated with `oasis setup -setup-update dynamic` to
> > minimize the amount of build-related stuff in the project):
> >
> > - Configure
> >
> >     % ocaml setup.ml -configure
> >
> >   This generates a file, `setup.data` in the current dir.  I've found no
> >   way to have it generated in a dedicated build dir.
> >
> > - Build
> >
> >     % ocaml setup.ml -build -build-dir /tmp/my_build_dir -no-links
> >
> >   This builds in the mentioned dir, but it also generates a `setup.log`
> >   in the current dir!
> >
> > I thought that `-C` could offer a way to achieve this, but it doesn't:
> >
> >     % mkdir /tmp/my_build_dir; cd /tmp/my_build_dir
> >     % ocaml /path/to/setup.ml -h
> >     Exception: Sys_error "_oasis: No such file or directory".
> >     File "/path/to/setup.ml", line 1:
> >     Error: Reference to undefined global `OASISDynRun'
> >
> > I'm not convinced `-C` *should* be the way to achieve what I want.
> >
> > Hopefully this makes it clearer what I mean.
> >
> >
> First, why not just copy everything in the new directory:
> % cp -R . /tmp/my_build_dir; cd /tmp/my_build_dir
> % ocaml setup.ml -configure
> % ocaml setup.ml -build
> 
> Maybe I miss some context to understand why this simple solution is not
> enough.
> 
> Going back at the other points. There are 3 categories of files:
> - the ones that are really generated at build time: setup.log, setup.data
>   -> they probably need to follow the indication provided -C. This is a bug
> - the ones that are generated by the underlying build system: _build
> (ocamlbuild)
>   -> as you wrote "-build -build-dir /tmp/my_build_dir -no-links" should be
> enough
> - the ones that are generated by the dynamic mode, which is a hack to
> prevent adding the files that other tools need:
>   -> these ones are tricky because they should have been in the source
> directory right from the beginning and being able to apply -C to them is
> equivalent to make sure that "ocamlfind"/"ocamlbuild"/other build tools
> will use -C and the source directory... This is clearly not trivial to
> assume that all tools will do.
> 
> My initial proposal to "cp" the files and build from the copy would
> probably solve your problem without assuming that all tools comply
> with -C. Tell me if it solves your problem.

Sure, that is a solution.

Maybe I was unclear to begin with.  I wanted a way to use *only*
oasis/buildtool to achieve my goal.  That is, I didn't want to involve
system-specific tools to copy the source from the project source dir
into the build dir before building. I couldn't find a way to use
oasis/buildtool to achieve that, so I sent an email to the list. *I*
consider the lack of a way to do that a deficiency in oasis/buildtool.

(Do note that it's completely acceptable to just say that this use case
isn't one that you see any value in and that the hack of copying the
source is the way to achieve a build that doesn't leave *anything*
behind in the source dir.)

/M

-- 
Magnus Therning                      OpenPGP: 0xAB4DFBA4 
email: magnus@therning.org   jabber: magnus@therning.org
twitter: magthe               http://therning.org/magnus

Failure is not an option. It comes bundled with the software.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [Caml-list] oasis building out-of-src and camlp4

[Sylvain Le Gall]

[-- Attachment #1: Type: text/plain, Size: 5462 bytes --]

Le jeu. 1 oct. 2015 à 09:23, Magnus Therning <magnus@therning.org> a écrit :

> On Thu, Oct 01, 2015 at 06:46:55AM +0000, Sylvain Le Gall wrote:
> > Hello,
> >
> > Le mer. 30 sept. 2015 à 13:49, Magnus Therning <magnus@therning.org> a
> > écrit :
> >
> > > On Tue, Sep 29, 2015 at 10:59:49PM +0000, Sylvain Le Gall wrote:
> > > > Please consider adding more comments to the following bug, which
> seems
> > > > close to what you are discussing:
> > > >
> > > >
> > >
> https://forge.ocamlcore.org/tracker/index.php?func=detail&aid=1473&group_id=54&atid=291
> > > >
> > > > I must admit that I totally miss the whole point of the discussion,
> > > > probably if you can make a summary of what you precisely need in the
> > > > BTS, I will be able to understand the problem better.
> > >
> > > Unfortunately I don't know enough about `oasis` to understand that bug
> > > report :(
> > >
> > > I'll try to explain how I'd like to use oasis/buildtool by looking how
> > > Haskell's Cabal lets me do it:
> > >
> > > - Configure without generating anything in the current dir:
> > >
> > >     % runhaskell Setup.hs --builddir /tmp/my_build_dir
> > >
> > > - Build using the just generated configuration, again without putting
> > >   anything in the current dir:
> > >
> > >     % runhaskell Setup.hs --builddir /tmp/my_build_dir
> > >
> > > Comparing this to oasis/buildtool (assuming I've a pre-existing
> > > `setup.ml` generated with `oasis setup -setup-update dynamic` to
> > > minimize the amount of build-related stuff in the project):
> > >
> > > - Configure
> > >
> > >     % ocaml setup.ml -configure
> > >
> > >   This generates a file, `setup.data` in the current dir.  I've found
> no
> > >   way to have it generated in a dedicated build dir.
> > >
> > > - Build
> > >
> > >     % ocaml setup.ml -build -build-dir /tmp/my_build_dir -no-links
> > >
> > >   This builds in the mentioned dir, but it also generates a `setup.log`
> > >   in the current dir!
> > >
> > > I thought that `-C` could offer a way to achieve this, but it doesn't:
> > >
> > >     % mkdir /tmp/my_build_dir; cd /tmp/my_build_dir
> > >     % ocaml /path/to/setup.ml -h
> > >     Exception: Sys_error "_oasis: No such file or directory".
> > >     File "/path/to/setup.ml", line 1:
> > >     Error: Reference to undefined global `OASISDynRun'
> > >
> > > I'm not convinced `-C` *should* be the way to achieve what I want.
> > >
> > > Hopefully this makes it clearer what I mean.
> > >
> > >
> > First, why not just copy everything in the new directory:
> > % cp -R . /tmp/my_build_dir; cd /tmp/my_build_dir
> > % ocaml setup.ml -configure
> > % ocaml setup.ml -build
> >
> > Maybe I miss some context to understand why this simple solution is not
> > enough.
> >
> > Going back at the other points. There are 3 categories of files:
> > - the ones that are really generated at build time: setup.log, setup.data
> >   -> they probably need to follow the indication provided -C. This is a
> bug
> > - the ones that are generated by the underlying build system: _build
> > (ocamlbuild)
> >   -> as you wrote "-build -build-dir /tmp/my_build_dir -no-links" should
> be
> > enough
> > - the ones that are generated by the dynamic mode, which is a hack to
> > prevent adding the files that other tools need:
> >   -> these ones are tricky because they should have been in the source
> > directory right from the beginning and being able to apply -C to them is
> > equivalent to make sure that "ocamlfind"/"ocamlbuild"/other build tools
> > will use -C and the source directory... This is clearly not trivial to
> > assume that all tools will do.
> >
> > My initial proposal to "cp" the files and build from the copy would
> > probably solve your problem without assuming that all tools comply
> > with -C. Tell me if it solves your problem.
>
> Sure, that is a solution.
>
> Maybe I was unclear to begin with.  I wanted a way to use *only*
> oasis/buildtool to achieve my goal.  That is, I didn't want to involve
> system-specific tools to copy the source from the project source dir
> into the build dir before building. I couldn't find a way to use
> oasis/buildtool to achieve that, so I sent an email to the list. *I*
> consider the lack of a way to do that a deficiency in oasis/buildtool.
>
> (Do note that it's completely acceptable to just say that this use case
> isn't one that you see any value in and that the hack of copying the
> source is the way to achieve a build that doesn't leave *anything*
> behind in the source dir.)
>
>
OASIS is mostly a glue system that mixes various systems, including system
tools. Under the hood, the setup.ml generated by OASIS will anyway use "cp"
or any ther system tools:
https://github.com/ocaml/oasis/blob/master/src/oasis/OASISFileUtil.ml#L124

I acknowledge that there is a value in your use case, but right now it is a
corner case for my personal usage. The hack should be enough to unblock
you. If you have an idea how to solve it without the hack, I will be glad
to merge a pull request:
https://github.com/ocaml/oasis

Regards
Sylvain


> /M
>
> --
> Magnus Therning                      OpenPGP: 0xAB4DFBA4
> email: magnus@therning.org   jabber: magnus@therning.org
> twitter: magthe               http://therning.org/magnus
>
> Failure is not an option. It comes bundled with the software.
>

[-- Attachment #2: Type: text/html, Size: 7813 bytes --]

Re: [Caml-list] oasis building out-of-src and camlp4

[Magnus Therning]

[-- Attachment #1: Type: text/plain, Size: 1034 bytes --]

On Fri, Oct 02, 2015 at 09:32:56AM +0000, Sylvain Le Gall wrote:

[...]

> OASIS is mostly a glue system that mixes various systems, including
> system tools. Under the hood, the setup.ml generated by OASIS will
> anyway use "cp" or any ther system tools:
> https://github.com/ocaml/oasis/blob/master/src/oasis/OASISFileUtil.ml#L124

Ah, I was under the impression that it was so tightly tied to the platform.

> I acknowledge that there is a value in your use case, but right now it
> is a corner case for my personal usage. The hack should be enough to
> unblock you. If you have an idea how to solve it without the hack, I
> will be glad to merge a pull request:
> https://github.com/ocaml/oasis

Fair enough :)

/M

-- 
Magnus Therning                      OpenPGP: 0xAB4DFBA4 
email: magnus@therning.org   jabber: magnus@therning.org
twitter: magthe               http://therning.org/magnus

A system is composed of components: a component is something you
understand.
     -- Professor Howard Aiken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [Caml-list] oasis building out-of-src and camlp4

[Anil Madhavapeddy]

On 22 Sep 2015, at 23:12, Magnus Therning <magnus@therning.org> wrote:
> 
> ~~~
> + /usr/bin/ocamldep.opt -modules src/main.ml > src/main.ml.depends
> File "src/main.ml", line 12, characters 4-8:
> Error: Syntax error
> Command exited with code 2.
> Compilation unsuccessful after building 1 target (0 cached) in 00:00:00.
> E: Failure("Command ''/usr/bin/ocamlbuild' src/main.byte -tag debug -build-dir /home/magnus/devo/tmp/oasis-camlp4-build/build' terminated with error code 10")
> ~~~
> 
> What am I missing here, how do I get the second example to build?

The ocamldep invocation here is not passing the -pp parameter to run
the source code through camlp4.  I'm not sure how you fix that in
CMake, but add the relevant lines (which should be the same as those
passed to ocamlc/ocamlopt) to ocamldep and it should get further...

regards,
Anil