[Caml-list] GODI is shutting down

[Caml-list] GODI is shutting down

[Gerd Stolpmann]

Dear subscriber,

Unfortunately, it is no longer possible for me to run the GODI
distribution. GODI will not upgrade to OCaml 4.01 once it is out, and it
will shut down the public service in the course of September 2013. The
website, camlcity.org, will remain up, but with reduced content. Existing
GODI installations can be continued to be used, but upgrades or bugfixes
will not be available when GODI is off.

Although there are still a lot of GODI users, it is unavoidable to shut
GODI down due to lack of supporters, especially package developers. I was
more or less alone in the past months, and my time contingent will not
allow it to do the upgrade to OCaml 4.01 alone (when it is released).

Also, there was a lot of noise about a competing packaging system for
OCaml in the past weeks: OPAM. Apparently, it got a lot of attention both
from individuals and from organizations. As I see it, the OCaml community
is too small to support two systems, and so in some sense GODI is
displaced by OPAM.

The sad part is that OPAM is only clearly better in one point, namely in
interacting with the community (via Github). In times where social
networks are worth billions this is probably the striking point. It
doesn't matter that OPAM lacks core functions like deleting all files when
a package is removed, and that it lacks many other features GODI has. So
there is some loss of functionality for the community (partly difficult to
replace, like GODI's support for Windows).

If somebody wants to take over GODI, please do so. The source code is
still available as well as the package directories. Maybe it is sufficient
to move the repository to a public place and to redesign the package
release process to give GODI a restart.

There is also another point that was driving me mad in the past weeks,
namely missing respect from the OPAM guys. Given the fact that OPAM is
only a thin layer around ocamlfind (and guess who wrote it), and given the
fact that GODI was pioneering in many fields, I was expecting nicer
wordings, and less dumb campaigning ("we have 400 packages, and you only
170"). OPAM is only harvesting what I seeded many years ago.

Let's hope these guys get now some kicks into their asses, and are forced
to add all the functionality to OPAM the OCaml community deserves.

Hoorn (NL), the 22nd July 2013,

Gerd Stolpmann
-- 
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de

Re: [Caml-list] GODI is shutting down

[Anil Madhavapeddy]

On 22 Jul 2013, at 22:21, "Gerd Stolpmann" <info@gerd-stolpmann.de> wrote:

> There is also another point that was driving me mad in the past weeks,
> namely missing respect from the OPAM guys. Given the fact that OPAM is
> only a thin layer around ocamlfind (and guess who wrote it), and given the
> fact that GODI was pioneering in many fields, I was expecting nicer
> wordings, and less dumb campaigning ("we have 400 packages, and you only
> 170"). OPAM is only harvesting what I seeded many years ago.

So when you describe the virtues of GODI, you call it 'advocating', and when
Fabrice describes the virtues of OPAM, it's 'dumb campaigning'. This is
splendidly reminiscent of US politics!

I don't find any of this source-based package management particularly novel.
I've worked in the OpenBSD ports tree since the late 90s, and it's way ahead
of either OPAM or GODI. I think where OPAM contributes most over any other
source manager are the three points on the front page that I iterated earlier
in this thread:

- multiple simultaneous compiler installations.
- flexible package constraints.
- a Git-friendly development workflow.

The first is very useful, but PATH munging is certainly not novel. The Git
workflow is heavily inspired by Homebrew.

The constraint solver has seen a tremendous amount of work though:
there's one version built into OPAM (tweaked relentlessly by Thomas with
heuristics), and the external CUDF solvers from the Mancoosi project which
are magically accurate (but not yet integrated by default until they are
packaged on different repositories).

Anyway, I'm not a GODI user, but I think you're doing a disservice to your
loyal GODI users if you shut it down in a huff. There are features (notably
Windows support) where it'll take OPAM some time to catch up.  I think that
the real reason that you're shutting it is nothing to do with "respect", but
is good old-fashioned burnt-out:

> Although there are still a lot of GODI users, it is unavoidable to shut GODI down due to lack of supporters, especially package developers. I was more or less alone in the past months, and my time contingent will not allow it to do the upgrade to OCaml 4.01 alone (when it is released).

Instead of doing such a heroic solo job (which I'm not being funny about --
it really is a lot of work), why not spend the time to open up GODI as well?

Most of the OPAM package descriptions come from *other people*. We just review
them, give feedback if necessary, click the merge button and fix errors that
creep through.  And even that takes up a surprising amount of time...

-anil

Re: [Caml-list] GODI is shutting down

[oliver]

On Mon, Jul 22, 2013 at 11:21:26PM +0200, Gerd Stolpmann wrote:
[...]
> The sad part is that OPAM is only clearly better in one point, namely in
> interacting with the community (via Github).
[...]

If this is the one and only problem, it might be solved,
by putting GODI on github.


Ciao,
   Oliver

Re: [Caml-list] GODI is shutting down

[Nicolas Braud-Santoni]

[-- Attachment #1: Type: text/plain, Size: 559 bytes --]

IMHO, there is more to OPAM's contributor-friendliness than "being on
GitHub".
But having a clear, simple process for contributing to OPAM may help.

My 2 zorkmids,
Nicolas

Le 7/22/13 7:44 PM, oliver a écrit :
> On Mon, Jul 22, 2013 at 11:21:26PM +0200, Gerd Stolpmann wrote:
> [...]
>> The sad part is that OPAM is only clearly better in one point, namely in
>> interacting with the community (via Github).
> [...]
>
> If this is the one and only problem, it might be solved,
> by putting GODI on github.
>
>
> Ciao,
>    Oliver
>



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [Caml-list] GODI is shutting down

[Francois Berenger]

Hello,

I wonder how many users of GODI on windows there are.

This is clearly the big plus over OPAM for the moment, I feel.

Regards,
F.

On 07/23/2013 06:21 AM, Gerd Stolpmann wrote:
> Dear subscriber,
>
> Unfortunately, it is no longer possible for me to run the GODI
> distribution. [...]

Re: [Caml-list] GODI is shutting down

[Gabriel Scherer]

On Tue, Jul 23, 2013 at 12:32 AM, Anil Madhavapeddy <anil@recoil.org> wrote:
> Instead of doing such a heroic solo job (which I'm not being funny about --
> it really is a lot of work), why not spend the time to open up GODI as well?
>
> Most of the OPAM package descriptions come from *other people*. We just review
> them, give feedback if necessary, click the merge button and fix errors that
> creep through.  And even that takes up a surprising amount of time...

To be fair, I never felt GODI to be a closed system. There are in fact
a reasonable number that were packaged by someone else than Gerd (see
http://godi.camlcity.org/godi/packages.html , the "released by" field
on packages; several were packaged by Virgile Prevosto, for example).
It's more that the packaging for GODI is a bit more difficult, and
probably just as importantly as there never was a strong community
dynamics to contribute to GODI packaging. It is now the norm that
everyone should write a META file for ocamlfind, and it is becoming
the norm to package for OPAM, but GODI probably came too early at a
time where developers expected distributions to do the packaging work
(which Debian and Fedora still do superbly), and there never was much
of a community dynamics for GODI packaging.

And implementation-wise, I have had the pleasure of interacting with
Gerd a couple of time to discuss possible changes to ocamlfind, and I
did not get the impression that the project was "closed"; I'm quite
sure anyone could contribute (or have contributed) to GODI with a
well-justified, good-looking patch.

On Tue, Jul 23, 2013 at 12:32 AM, Anil Madhavapeddy <anil@recoil.org> wrote:
> On 22 Jul 2013, at 22:21, "Gerd Stolpmann" <info@gerd-stolpmann.de> wrote:
>
>> There is also another point that was driving me mad in the past weeks,
>> namely missing respect from the OPAM guys. Given the fact that OPAM is
>> only a thin layer around ocamlfind (and guess who wrote it), and given the
>> fact that GODI was pioneering in many fields, I was expecting nicer
>> wordings, and less dumb campaigning ("we have 400 packages, and you only
>> 170"). OPAM is only harvesting what I seeded many years ago.
>
> So when you describe the virtues of GODI, you call it 'advocating', and when
> Fabrice describes the virtues of OPAM, it's 'dumb campaigning'. This is
> splendidly reminiscent of US politics!
>
> I don't find any of this source-based package management particularly novel.
> I've worked in the OpenBSD ports tree since the late 90s, and it's way ahead
> of either OPAM or GODI. I think where OPAM contributes most over any other
> source manager are the three points on the front page that I iterated earlier
> in this thread:
>
> - multiple simultaneous compiler installations.
> - flexible package constraints.
> - a Git-friendly development workflow.
>
> The first is very useful, but PATH munging is certainly not novel. The Git
> workflow is heavily inspired by Homebrew.
>
> The constraint solver has seen a tremendous amount of work though:
> there's one version built into OPAM (tweaked relentlessly by Thomas with
> heuristics), and the external CUDF solvers from the Mancoosi project which
> are magically accurate (but not yet integrated by default until they are
> packaged on different repositories).
>
> Anyway, I'm not a GODI user, but I think you're doing a disservice to your
> loyal GODI users if you shut it down in a huff. There are features (notably
> Windows support) where it'll take OPAM some time to catch up.  I think that
> the real reason that you're shutting it is nothing to do with "respect", but
> is good old-fashioned burnt-out:
>
>> Although there are still a lot of GODI users, it is unavoidable to shut GODI down due to lack of supporters, especially package developers. I was more or less alone in the past months, and my time contingent will not allow it to do the upgrade to OCaml 4.01 alone (when it is released).
>
> Instead of doing such a heroic solo job (which I'm not being funny about --
> it really is a lot of work), why not spend the time to open up GODI as well?
>
> Most of the OPAM package descriptions come from *other people*. We just review
> them, give feedback if necessary, click the merge button and fix errors that
> creep through.  And even that takes up a surprising amount of time...
>
> -anil
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] GODI is shutting down

[Keyan]

Just to through in my 2 cents. I didn't understand what GODI did, when I first tried it. Therefore, I sticked with my OS package manage for a long time (home-brew on Mac OS X). The reason I now use OPAM, it worked straight out of the box, and I know exactly what is happening. The biggest benefit: there is a clean and easy way to uninstall everything (rm -rf ~/.opam), which is the killer argument for me.

This is not intended to heat up the discussion, but to explain, from a regular user perspective, why I chose opam over godi.

Cheers,
Keyan


On 23 Jul 2013, at 08:49, Gabriel Scherer <gabriel.scherer@gmail.com> wrote:

> On Tue, Jul 23, 2013 at 12:32 AM, Anil Madhavapeddy <anil@recoil.org> wrote:
>> Instead of doing such a heroic solo job (which I'm not being funny about --
>> it really is a lot of work), why not spend the time to open up GODI as well?
>> 
>> Most of the OPAM package descriptions come from *other people*. We just review
>> them, give feedback if necessary, click the merge button and fix errors that
>> creep through.  And even that takes up a surprising amount of time...
> 
> To be fair, I never felt GODI to be a closed system. There are in fact
> a reasonable number that were packaged by someone else than Gerd (see
> http://godi.camlcity.org/godi/packages.html , the "released by" field
> on packages; several were packaged by Virgile Prevosto, for example).
> It's more that the packaging for GODI is a bit more difficult, and
> probably just as importantly as there never was a strong community
> dynamics to contribute to GODI packaging. It is now the norm that
> everyone should write a META file for ocamlfind, and it is becoming
> the norm to package for OPAM, but GODI probably came too early at a
> time where developers expected distributions to do the packaging work
> (which Debian and Fedora still do superbly), and there never was much
> of a community dynamics for GODI packaging.
> 
> And implementation-wise, I have had the pleasure of interacting with
> Gerd a couple of time to discuss possible changes to ocamlfind, and I
> did not get the impression that the project was "closed"; I'm quite
> sure anyone could contribute (or have contributed) to GODI with a
> well-justified, good-looking patch.
> 
> On Tue, Jul 23, 2013 at 12:32 AM, Anil Madhavapeddy <anil@recoil.org> wrote:
>> On 22 Jul 2013, at 22:21, "Gerd Stolpmann" <info@gerd-stolpmann.de> wrote:
>> 
>>> There is also another point that was driving me mad in the past weeks,
>>> namely missing respect from the OPAM guys. Given the fact that OPAM is
>>> only a thin layer around ocamlfind (and guess who wrote it), and given the
>>> fact that GODI was pioneering in many fields, I was expecting nicer
>>> wordings, and less dumb campaigning ("we have 400 packages, and you only
>>> 170"). OPAM is only harvesting what I seeded many years ago.
>> 
>> So when you describe the virtues of GODI, you call it 'advocating', and when
>> Fabrice describes the virtues of OPAM, it's 'dumb campaigning'. This is
>> splendidly reminiscent of US politics!
>> 
>> I don't find any of this source-based package management particularly novel.
>> I've worked in the OpenBSD ports tree since the late 90s, and it's way ahead
>> of either OPAM or GODI. I think where OPAM contributes most over any other
>> source manager are the three points on the front page that I iterated earlier
>> in this thread:
>> 
>> - multiple simultaneous compiler installations.
>> - flexible package constraints.
>> - a Git-friendly development workflow.
>> 
>> The first is very useful, but PATH munging is certainly not novel. The Git
>> workflow is heavily inspired by Homebrew.
>> 
>> The constraint solver has seen a tremendous amount of work though:
>> there's one version built into OPAM (tweaked relentlessly by Thomas with
>> heuristics), and the external CUDF solvers from the Mancoosi project which
>> are magically accurate (but not yet integrated by default until they are
>> packaged on different repositories).
>> 
>> Anyway, I'm not a GODI user, but I think you're doing a disservice to your
>> loyal GODI users if you shut it down in a huff. There are features (notably
>> Windows support) where it'll take OPAM some time to catch up.  I think that
>> the real reason that you're shutting it is nothing to do with "respect", but
>> is good old-fashioned burnt-out:
>> 
>>> Although there are still a lot of GODI users, it is unavoidable to shut GODI down due to lack of supporters, especially package developers. I was more or less alone in the past months, and my time contingent will not allow it to do the upgrade to OCaml 4.01 alone (when it is released).
>> 
>> Instead of doing such a heroic solo job (which I'm not being funny about --
>> it really is a lot of work), why not spend the time to open up GODI as well?
>> 
>> Most of the OPAM package descriptions come from *other people*. We just review
>> them, give feedback if necessary, click the merge button and fix errors that
>> creep through.  And even that takes up a surprising amount of time...
>> 
>> -anil
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

AW: [Caml-list] GODI is shutting down

[Gerd Stolpmann]

Am 23.07.2013 10:46:41 schrieb(en) Keyan:
> Just to through in my 2 cents. I didn't understand what GODI did,  
> when I first tried it. Therefore, I sticked with my OS package manage  
> for a long time (home-brew on Mac OS X). The reason I now use OPAM,  
> it worked straight out of the box, and I know exactly what is  
> happening. The biggest benefit: there is a clean and easy way to  
> uninstall everything (rm -rf ~/.opam), which is the killer argument  
> for me.
> 
> This is not intended to heat up the discussion, but to explain, from  
> a regular user perspective, why I chose opam over godi.

This is just misinformation, and no argument, because godi also  
installs into a separate directory hierarchy. You are a victim of  
OPAM's campaign.

Gerd

> 
> Cheers,
> Keyan
> 
> 
> On 23 Jul 2013, at 08:49, Gabriel Scherer <gabriel.scherer@gmail.com>  
> wrote:
> 
> > On Tue, Jul 23, 2013 at 12:32 AM, Anil Madhavapeddy  
> <anil@recoil.org> wrote:
> >> Instead of doing such a heroic solo job (which I'm not being funny  
> about --
> >> it really is a lot of work), why not spend the time to open up  
> GODI as well?
> >>
> >> Most of the OPAM package descriptions come from *other people*. We  
> just review
> >> them, give feedback if necessary, click the merge button and fix  
> errors that
> >> creep through.  And even that takes up a surprising amount of  
> time...
> >
> > To be fair, I never felt GODI to be a closed system. There are in  
> fact
> > a reasonable number that were packaged by someone else than Gerd  
> (see
> > http://godi.camlcity.org/godi/packages.html , the "released by"  
> field
> > on packages; several were packaged by Virgile Prevosto, for  
> example).
> > It's more that the packaging for GODI is a bit more difficult, and
> > probably just as importantly as there never was a strong community
> > dynamics to contribute to GODI packaging. It is now the norm that
> > everyone should write a META file for ocamlfind, and it is becoming
> > the norm to package for OPAM, but GODI probably came too early at a
> > time where developers expected distributions to do the packaging  
> work
> > (which Debian and Fedora still do superbly), and there never was  
> much
> > of a community dynamics for GODI packaging.
> >
> > And implementation-wise, I have had the pleasure of interacting with
> > Gerd a couple of time to discuss possible changes to ocamlfind, and  
> I
> > did not get the impression that the project was "closed"; I'm quite
> > sure anyone could contribute (or have contributed) to GODI with a
> > well-justified, good-looking patch.
> >
> > On Tue, Jul 23, 2013 at 12:32 AM, Anil Madhavapeddy  
> <anil@recoil.org> wrote:
> >> On 22 Jul 2013, at 22:21, "Gerd Stolpmann"  
> <info@gerd-stolpmann.de> wrote:
> >>
> >>> There is also another point that was driving me mad in the past  
> weeks,
> >>> namely missing respect from the OPAM guys. Given the fact that  
> OPAM is
> >>> only a thin layer around ocamlfind (and guess who wrote it), and  
> given the
> >>> fact that GODI was pioneering in many fields, I was expecting  
> nicer
> >>> wordings, and less dumb campaigning ("we have 400 packages, and  
> you only
> >>> 170"). OPAM is only harvesting what I seeded many years ago.
> >>
> >> So when you describe the virtues of GODI, you call it  
> 'advocating', and when
> >> Fabrice describes the virtues of OPAM, it's 'dumb campaigning'.  
> This is
> >> splendidly reminiscent of US politics!
> >>
> >> I don't find any of this source-based package management  
> particularly novel.
> >> I've worked in the OpenBSD ports tree since the late 90s, and it's  
> way ahead
> >> of either OPAM or GODI. I think where OPAM contributes most over  
> any other
> >> source manager are the three points on the front page that I  
> iterated earlier
> >> in this thread:
> >>
> >> - multiple simultaneous compiler installations.
> >> - flexible package constraints.
> >> - a Git-friendly development workflow.
> >>
> >> The first is very useful, but PATH munging is certainly not novel.  
> The Git
> >> workflow is heavily inspired by Homebrew.
> >>
> >> The constraint solver has seen a tremendous amount of work though:
> >> there's one version built into OPAM (tweaked relentlessly by  
> Thomas with
> >> heuristics), and the external CUDF solvers from the Mancoosi  
> project which
> >> are magically accurate (but not yet integrated by default until  
> they are
> >> packaged on different repositories).
> >>
> >> Anyway, I'm not a GODI user, but I think you're doing a disservice  
> to your
> >> loyal GODI users if you shut it down in a huff. There are features  
> (notably
> >> Windows support) where it'll take OPAM some time to catch up.  I  
> think that
> >> the real reason that you're shutting it is nothing to do with  
> "respect", but
> >> is good old-fashioned burnt-out:
> >>
> >>> Although there are still a lot of GODI users, it is unavoidable  
> to shut GODI down due to lack of supporters, especially package  
> developers. I was more or less alone in the past months, and my time  
> contingent will not allow it to do the upgrade to OCaml 4.01 alone  
> (when it is released).
> >>
> >> Instead of doing such a heroic solo job (which I'm not being funny  
> about --
> >> it really is a lot of work), why not spend the time to open up  
> GODI as well?
> >>
> >> Most of the OPAM package descriptions come from *other people*. We  
> just review
> >> them, give feedback if necessary, click the merge button and fix  
> errors that
> >> creep through.  And even that takes up a surprising amount of  
> time...
> >>
> >> -anil
> >> --
> >> Caml-list mailing list.  Subscription management and archives:
> >> https://sympa.inria.fr/sympa/arc/caml-list
> >> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> >> Bug reports: http://caml.inria.fr/bin/caml-bugs
> >
> > --
> > Caml-list mailing list.  Subscription management and archives:
> > https://sympa.inria.fr/sympa/arc/caml-list
> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> > Bug reports: http://caml.inria.fr/bin/caml-bugs
> 
> 
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 



-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Caml-list] GODI is shutting down

[Adrien Nader]

Hi,

On Mon, Jul 22, 2013, Gerd Stolpmann wrote:
> Dear subscriber,
> 
> Unfortunately, it is no longer possible for me to run the GODI
> distribution. GODI will not upgrade to OCaml 4.01 once it is out, and it
> will shut down the public service in the course of September 2013. The
> website, camlcity.org, will remain up, but with reduced content. Existing
> GODI installations can be continued to be used, but upgrades or bugfixes
> will not be available when GODI is off.
> 
> Although there are still a lot of GODI users, it is unavoidable to shut
> GODI down due to lack of supporters, especially package developers. I was
> more or less alone in the past months, and my time contingent will not
> allow it to do the upgrade to OCaml 4.01 alone (when it is released).

It is very sad to hear that. I was expecting some clashes for the past
few months so I'm not completely surprised.
I've been meaning to do more for godi but I've been completely unable to
do so unfortunately and I'm sorry for that.

> Also, there was a lot of noise about a competing packaging system for
> OCaml in the past weeks: OPAM. Apparently, it got a lot of attention both
> from individuals and from organizations. As I see it, the OCaml community
> is too small to support two systems, and so in some sense GODI is
> displaced by OPAM.

I believe that this is the same as with LLVM and GCC. Two years ago,
llvm was supposed to beat GCC within a year: performance of the
generated code was increasing very quickly.
It was a new software, not hampered by decades of development, and the
rule of 80/20 struck. It was fairly obvious that the rate of
improvements would slow down and that people would start asking for more
stability and compatibility (to this day, llvm still doesn't have minor
releases).

It must have been hell for GCC people back then. Noone cared that they
were more compatible (for hosts, targets, options and combinations of
these), more stable, generated faster code, had been around for decades,
...

> The sad part is that OPAM is only clearly better in one point, namely in
> interacting with the community (via Github). In times where social
> networks are worth billions this is probably the striking point. It
> doesn't matter that OPAM lacks core functions like deleting all files when
> a package is removed, and that it lacks many other features GODI has. So
> there is some loss of functionality for the community (partly difficult to
> replace, like GODI's support for Windows).

To be honest, I've never understood why opam was "started". I believe
the time would have been much better spent improving godi.
I've heard complaints against the UI and the package release process of
godi.
How can such complaints about the UI be a reason to start a new
project?! Couldn't the package release process be improved gradually?

As for github, I concur its best feature is the "social" thing. And
sometimes it gets in the way: you can see people who've forked your
repo and their commits easily. But for other things, cgit is way better
and clearer. Sure, it doesn't have a shiny CSS. On the other hand, it
doesn't put flash objects in webpages like github.

> If somebody wants to take over GODI, please do so. The source code is
> still available as well as the package directories. Maybe it is sufficient
> to move the repository to a public place and to redesign the package
> release process to give GODI a restart.

You haven't only written the code but you've also setup a whole
infrastructure. Would you mind trying to pass it over or migrate it,
maybe not leaving completely but letting others do some of the annoying
admin-related tasks?

> There is also another point that was driving me mad in the past weeks,
> namely missing respect from the OPAM guys. Given the fact that OPAM is
> only a thin layer around ocamlfind (and guess who wrote it), and given the
> fact that GODI was pioneering in many fields, I was expecting nicer
> wordings, and less dumb campaigning ("we have 400 packages, and you only
> 170"). OPAM is only harvesting what I seeded many years ago.

I tend to agree here. Something like 5 years ago, management of OCaml
library was a real pain. Ocamlfind support was everything but universal
(I remember when ocamlfind-enabled build systems where the minority and
I found them annoying). I believe godi really helped with that but it
also had to support build systems not using ocamlfind and that made some
godi packages more complicated.

The people involved in OPAM have clearly been more vocal and I'm
definitely not going to blame them for that: if you build a new project
but don't advertise it, noone will use it. Some of the messages were
probably lacking some tact however.


As far as I'm concerned, I don't want to use OPAM. I've been happy with
GODI and would really like to continue to use it. I do not want to use a
high-speed source-based rolling-release system and I believe that it is
bad, especially for beginners who need a stable environment.

I've been very irritated by pieces of advice for beginners that
mentionned using svn branches of the OCaml compiler. OPAM makes it easy;
great! It's still an unreleased version that is still in development.
Generally speaking, I find the use of bleeding-edge software bad. It
might be good OCaml libraries that are well-tested but it doesn't
matter: you're going to get into troubles sooner or later.

I'm not advocating waiting for years for things to settle but vim
$package, :%s/0.14/0.15/g, :wq, git commit -a -m 'foo: Update to 0.15.',
git push might be too quick.

I also believe "pinning" is a bad idea in general: it picks bits and
pieces of software from different timeframes. It will work for
exceptional cases but otherwise you end up with an exponential explosion
of combinations of versions that haven't been tested together.

There could be a git branch or a git repo for stable stuff along with
the development tree. Citrix already does that and others already do
that too. Which means that there are several parallel distributions and
that has its drawbacks too.

Obviously, all of these can be fixed or improved, through code, process
and probably better: a combination of both. However, currently, there
are still open issues.

I've lacked time recently and I'm still using godi with 3.12.1. And I
can still update it and it works as well as the 4.00 branch. I doubt
that I could get the same with OPAM (had it started with 3.12).

-- 
Adrien Nader

[Caml-list] Re: [Godi-list] GODI is shutting down

[Thomas Gazagnaire]

> There is also another point that was driving me mad in the past weeks,
> namely missing respect from the OPAM guys. Given the fact that OPAM is
> only a thin layer around ocamlfind (and guess who wrote it), and given the
> fact that GODI was pioneering in many fields, I was expecting nicer
> wordings, and less dumb campaigning ("we have 400 packages, and you only
> 170"). OPAM is only harvesting what I seeded many years ago.

I am sorry you feel that way, but I have to correct some errors. First, you seem to not make the difference between exposing a fact: "OPAM packaging workflow is easier than GODI's, hence the higher number of packages" and being disrespectful: (insert here any of the sentence related to OPAM in your recent emails).

Second, OPAM is *not* a thin layer around ocamlfind: it is a (not-so-thin) layer around the dose3 library (used by the Debian tools to manage complex dependencies), around heterogeneous metadata backends (HTTP, rsync, Git, hg, darcs) and compiler environment variables (to support multiple switch). OPAM is build-system agnostic (and could be used to build C or C++ projects for instance) and its initial goal was to integrate nicely with the existing ecosystem, namely ocamlbuild, ocamlfind and oasis (and omake and ocp-build). So yes, people are using OPAM and ocamlfind together, and this is a very good thing -- and they can use any other system if they like it.

We have been focusing on the final user experience, based on our the constant feedback from the community and industrial users, and lot of people seem to appreciate the effort. I am very sorry you don't.
 
Best,
Thomas

AW: [Caml-list] GODI is shutting down

[Gerd Stolpmann]

Am 23.07.2013 00:32:43 schrieb(en) Anil Madhavapeddy:
> On 22 Jul 2013, at 22:21, "Gerd Stolpmann" <info@gerd-stolpmann.de>  
> wrote:
> 
> > There is also another point that was driving me mad in the past  
> weeks,
> > namely missing respect from the OPAM guys. Given the fact that OPAM  
> is
> > only a thin layer around ocamlfind (and guess who wrote it), and  
> given the
> > fact that GODI was pioneering in many fields, I was expecting nicer
> > wordings, and less dumb campaigning ("we have 400 packages, and you  
> only
> > 170"). OPAM is only harvesting what I seeded many years ago.
> 
> So when you describe the virtues of GODI, you call it 'advocating',  
> and when
> Fabrice describes the virtues of OPAM, it's 'dumb campaigning'. This  
> is
> splendidly reminiscent of US politics!

I do not live in the US, so I have no idea what you are talking about.

> Anyway, I'm not a GODI user, but I think you're doing a disservice to  
> your
> loyal GODI users if you shut it down in a huff. There are features  
> (notably
> Windows support) where it'll take OPAM some time to catch up.  I  
> think that
> the real reason that you're shutting it is nothing to do with  
> "respect", but
> is good old-fashioned burnt-out:

If you mean that I'm alone not louder than the crowd, maybe call it  
burnout. At a certain point I noticed that arguments do not count  
anymore, and it doesn't matter what I do. So it doesn't make sense to  
put more energy into this project.

> > Although there are still a lot of GODI users, it is unavoidable to  
> shut GODI down due to lack of supporters, especially package  
> developers. I was more or less alone in the past months, and my time  
> contingent will not allow it to do the upgrade to OCaml 4.01 alone  
> (when it is released).
> 
> Instead of doing such a heroic solo job (which I'm not being funny  
> about --
> it really is a lot of work), why not spend the time to open up GODI  
> as well?
> 
> Most of the OPAM package descriptions come from *other people*. We  
> just review
> them, give feedback if necessary, click the merge button and fix  
> errors that
> creep through.  And even that takes up a surprising amount of time...

That's again campaign - you have absolutely no idea, but the reader of  
your lines will have the impression that GODI is a closed system only  
created by a single guy. Actually, most GODI packages were done by  
other people (well, initially at least, but many packages got orphaned  
over time). GODI is an open system, only that it's not as radically  
transparent as people are nowadays used to (i.e. there is no web site  
with user statistics, no public commit logs, no easy way to pull  
commits from other systems, no whatever other bells and whistles, and  
you needed to send me an email to register, old style). But the users  
had all freedom, including doing releases completely on their own (no  
pull request bureaucracy).

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Caml-list] GODI is shutting down

[Paolo Donadeo]

[-- Attachment #1: Type: text/plain, Size: 280 bytes --]

On Tue, Jul 23, 2013 at 10:57 AM, Gerd Stolpmann <info@gerd-stolpmann.de>wrote:

> This is just misinformation, and no argument, because godi also installs
> into a separate directory hierarchy.


The equivalent is rm -Rf /opt/godi, if you had installed GODI there.


-- 
*Paolo*

[-- Attachment #2: Type: text/html, Size: 637 bytes --]

Re: [Caml-list] GODI is shutting down

[Jesper Louis Andersen]

[-- Attachment #1: Type: text/plain, Size: 1365 bytes --]

On Tue, Jul 23, 2013 at 11:34 AM, Gerd Stolpmann <info@gerd-stolpmann.de>wrote:

> But the users had all freedom, including doing releases completely on
> their own (no pull request bureaucracy).


The nice thing about a Git-based solution is that you can just build your
own fork if you want. So you don't have to wait on PRs to fall through,
thought it is certainly better for integration in the longer run to merge
and converge on a single repository. The key thing is that submitting a PR
is actually hard work for the contributor, but easy work for the
integrator. The idea is to make the work of the integrator easy since
he/she is the bottleneck.

I think the central part here is that managing package repositories is
*hard work*. There is a lot of things happening all the time and ensuring
dependency convergence among packages can also be pretty nasty if the
dependency chains grow deep. Yes, you can install multiple versions of
packages, but that does not avoid a common diamond problem of convergence.

Personally, I think the pragmatic action is to rally on one packaging
system. Ocaml is way way too small a community to do otherwise. Said
packaging system may be good or bad, but that is less important. You may
also be a victim of an unfair PR-campaign, but really, such is life in
communities. Technical merit is only part of the whole.


-- 
J.

[-- Attachment #2: Type: text/html, Size: 1846 bytes --]

AW: [Caml-list] GODI is shutting down

[Gerd Stolpmann]

> You haven't only written the code but you've also setup a whole
> infrastructure. Would you mind trying to pass it over or migrate it,
> maybe not leaving completely but letting others do some of the  
> annoying
> admin-related tasks?

Both is possible - there are two VMs, one with the repository and the  
release infrastructure, one with the autobuilder, and it would be easy  
to migrate it somewhere else. It's also possible to leave it where it  
is if only someone else takes over the leadership of the project, and  
invests time to renew it. (My problem is only that I don't have the  
energy anymore to run it, but if I can help with computing resources it  
would be a pleasure.)

> I also believe "pinning" is a bad idea in general: it picks bits and
> pieces of software from different timeframes. It will work for
> exceptional cases but otherwise you end up with an exponential  
> explosion
> of combinations of versions that haven't been tested together.

Well, there is in deed one good use: imagine you have a group of people  
needing to use exactly the same versions. So you basically freeze the  
versions at one point in time, knowing that they work well together. If  
you need newer versions, you move the "pin" together (or update the  
same set of local patches). Ideally, you can also create fresh  
installation with the pinned versions. GODI supports this for some time  
now (although by editing files only, but I guess this is ok for a  
group, say a team in a company).

> There could be a git branch or a git repo for stable stuff along with
> the development tree. Citrix already does that and others already do
> that too. Which means that there are several parallel distributions  
> and
> that has its drawbacks too.

Yes, you can emulate this with git.

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: [Caml-list] GODI is shutting down

[oliver]

On Tue, Jul 23, 2013 at 11:07:13AM +0200, Adrien Nader wrote:
[...]
> There could be a git branch or a git repo for stable stuff along with
> the development tree.
[...]

Let each contributing developer get it's own devel-branch,
then selectively merging certain branches to the main branch ("master")
should be easy to incorporate changes that are fine, and the ones that
seem to be problematic, can possibly talked about and adapted, before
merged in.
Permissions should be used accordingly, to avoid a mess.

I think using git (with or without github) is making it much much
more easy than using email.
It needs some work to configure/setup the environment first; but then it's fun.

Ciao,
   Oliver

Re: [Caml-list] GODI is shutting down

[Marek Kubica]

On Tue, 23 Jul 2013 10:57:12 +0200
Gerd Stolpmann <info@gerd-stolpmann.de> wrote:

> This is just misinformation, and no argument, because godi also  
> installs into a separate directory hierarchy. You are a victim of  
> OPAM's campaign.

You keep on citing a campaign, but I haven't seen a campaign. I have
seen a number of blog posts, announcements on OPAM. It is not that there
was a street mob advocating against using GODI.

It is just that OPAM seemed to be a good deal more active. Also, the
OPAM-folks even in this thread seemed very reasonable and nice and did
not bash GODI at all, so I have no idea where this comes from. I think
you are doing your efforts a disservice by flaming the OPAM
maintainers, who just like you, put in a lot of work to improve the
OCaml packaging situation.

regards,
Marek

Re: [Caml-list] Re: [Godi-list] GODI is shutting down

[Pierre-Etienne Meunier]

Hi,

I'm only using my os' package manager at this time, so I am not part of the troll.
But I find that Gerd made an interesting point, a few mails ago:

> So far I've removed Core from GODI because it misses QA standards, and
> have no plans to add it again.
> 
> Gerd

A part of the troll has consisted of arguments about github, which is a non-open-source centralized version of what we had before. I you create a repository to distribute a cool open source replacement of github, with enough extra features to kiil them in one week, and an innovative system that pays you to host your content, I believe the people at github have the right to shut your repository down.

It seems that neither godi nor opam do any better (for this particular point, of course): the dictator is different in the three cases, but there is a dictator. In the case of opam, he has a friend dictator (github).

Will we see opam become distributed? It would be awesome to be able to send packages back and forth between the repositories, without any centralized decision maker, just as in darcs, for instance. Ocaml has this property of being an atomic unix tool, conceived in this way, right?

Cheers,

Pierre

Re: [Caml-list] Re: [Godi-list] GODI is shutting down

[David Sheets]

On Tue, Jul 23, 2013 at 4:32 PM, Pierre-Etienne Meunier
<pierreetienne.meunier@gmail.com> wrote:
> Hi,
>
> I'm only using my os' package manager at this time, so I am not part of the troll.
> But I find that Gerd made an interesting point, a few mails ago:
>
>> So far I've removed Core from GODI because it misses QA standards, and
>> have no plans to add it again.
>>
>> Gerd
>
> A part of the troll has consisted of arguments about github, which is a non-open-source centralized version of what we had before. I you create a repository to distribute a cool open source replacement of github, with enough extra features to kiil them in one week, and an innovative system that pays you to host your content, I believe the people at github have the right to shut your repository down.

OPAM uses git. git is distributed. Github is a tool that you can (and
they do) use with git.

> It seems that neither godi nor opam do any better (for this particular point, of course): the dictator is different in the three cases, but there is a dictator. In the case of opam, he has a friend dictator (github).

There is no required dictator with OPAM. You may create your own
git-based opam repository and use it with your own package
descriptions in your own namespace. It is very flexible.

> Will we see opam become distributed? It would be awesome to be able to send packages back and forth between the repositories, without any centralized decision maker, just as in darcs, for instance. Ocaml has this property of being an atomic unix tool, conceived in this way, right?

You can do this, today. Simply send an email patch from your local
opam repository to your friends.

Hope this helps,

David

Re: [Caml-list] Re: [Godi-list] GODI is shutting down

[Daniel Bünzli]

Le mardi, 23 juillet 2013 à 16:32, Pierre-Etienne Meunier a écrit :
> It seems that neither godi nor opam do any better (for this particular point, of course): the dictator is different in the three cases, but there is a dictator. In the case of opam, he has a friend dictator (github).

I can't speak for godi, but at least for opam you get your facts wrong. There's nothing specific about github in opam and everybody can be a dictator. I'm one, just type:

    opam repo add erratique-u http://erratique.ch/software/opam/unreleased  
    opam upgrade  


if you want to be one of my subjects.

Best,

Daniel

Re: [Caml-list] Re: [Godi-list] GODI is shutting down

[Pierre-Etienne Meunier]

> I can't speak for godi, but at least for opam you get your facts wrong. There's nothing specific about github in opam and everybody can be a dictator. I'm one, just type:
> 
>    opam repo add erratique-u http://erratique.ch/software/opam/unreleased  
>    opam upgrade  
> 
> 
> if you want to be one of my subjects.


This is awesome! Is there a simple documentation somewhere? What made me think there was a dictator is the fact that the documentation only mentions "*the* official repository".

My apologies for the misunderstanding.

Pierre

Re: [Caml-list] Re: [Godi-list] GODI is shutting down

[Ashish Agarwal]

[-- Attachment #1: Type: text/plain, Size: 273 bytes --]

On Tue, Jul 23, 2013 at 12:19 PM, Pierre-Etienne Meunier <
pierreetienne.meunier@gmail.com> wrote:

Is there a simple documentation somewhere?


opam repository --help

Note the `remove` and `add` options. There is nothing special about the
default one. You can remove it.

[-- Attachment #2: Type: text/html, Size: 599 bytes --]

Re: [Caml-list] Re: [Godi-list] GODI is shutting down

[Daniel Bünzli]

Le mardi, 23 juillet 2013 à 17:19, Pierre-Etienne Meunier a écrit :
> This is awesome! Is there a simple documentation somewhere? What made me think there was a dictator is the fact that the documentation only mentions "*the* official repository".


The packaging section [1] of opam's docs will tell you how to make a local repo. Other than that this thread [2] tells you how to make an http repo.  

Best,

Daniel

[1] http://opam.ocamlpro.com/doc/Packaging.html
[2] https://sympa.inria.fr/sympa/arc/caml-list/2013-07/msg00011.html

Re: [Caml-list] Re: [Godi-list] GODI is shutting down

[Virgile Prevosto]

Hello,

2013/7/23 Daniel Bünzli <daniel.buenzli@erratique.ch>:
> I can't speak for godi, but at least for opam you get your facts wrong. There's nothing specific about github in opam and everybody can be a dictator. I'm one, just type:
>
Just for the sack of completeness, as I feel that this flame war is a
little bit one-sided, this is also perfectly doable with Godi (like
most of the awesome opam features that have been mentioned here these
last few days by the way), either by changing the value of the
GODI_BUILD_SITE environment variable (yes, I know, it's a bit
old-fashioned), and/or with the profiles that have appeared in the
latest version of Godi.

Best regards,
--
E tutto per oggi, a la prossima volta
Virgile

Re: [Caml-list] Re: [Godi-list] GODI is shutting down

[Yaron Minsky]

On Tue, Jul 23, 2013 at 11:32 AM, Pierre-Etienne Meunier  wrote:
> Hi,
>
> I'm only using my os' package manager at this time, so I am not part of the troll.
> But I find that Gerd made an interesting point, a few mails ago:
>
>> So far I've removed Core from GODI because it misses QA standards, and
>> have no plans to add it again.

I was not planning on responding substantively to Gerd's point about
Core, but I don't want to let a false impression stand.  Core, Async,
Typeconv, Sexplib and Binprot are all high-quality codebases that
benefit from quite a bit of testing, code review, and care.  I'm not
sure what Gerd is referring to, but it is true that we stopped putting
effort into packaging for Godi some time ago (You can look here
http://bit.ly/13aPCEI for the history behind why we stopped using GODI
ourselves), but our releases are at this point quite smooth.  We have
a reliable weekly release with tarballs, opam packages, and exports
into git (hosted at github) and mercurial (hosted at bitbucket).

Surely there are things to improve (documentation, for example), but I
don't think there's a general QA problem.

As a general point, I do think that a weekly release is not what most
people want or need.  That's why OCaml Labs and OCamlPro are working
on an OCaml Platform that contains stable versions of many important
libraries, with OPAM used to manage those packages.  I think most
programmers would be happier using such a platform release, rather
than a bleeding-edge weekly one.

y

Re: [Caml-list] Re: [Godi-list] GODI is shutting down

[Pierre-Etienne Meunier]

Em 23/07/2013, às 12:38, Yaron Minsky escreveu:

> On Tue, Jul 23, 2013 at 11:32 AM, Pierre-Etienne Meunier  wrote:
>> Hi,
>> 
>> I'm only using my os' package manager at this time, so I am not part of the troll.
>> But I find that Gerd made an interesting point, a few mails ago:
>> 
>>> So far I've removed Core from GODI because it misses QA standards, and
>>> have no plans to add it again.
> 
> I was not planning on responding substantively to Gerd's point about
> Core, but I don't want to let a false impression stand.


I am sorry if I did not make myself clear: I was not trying to say anything about Core, nor Jane Street, nor you!

My point is, I am scared by internet dictators, and, whether or not Core is a great library, I believe that no one, except maybe its authors, should have the right to remove it from this fantastic open public space that the internet is, and prevent its distribution. This is also what scares me when "github-friendly" is given as an argument for something.

Pierre


ps: IMHO, as one of its occasional users, Core *is* a great library. Thanks for sharing this colossal and amazing amount of work, Yaron.

Re: [Caml-list] Re: [Godi-list] GODI is shutting down

[Yaron Minsky]

My apologies for misunderstanding your post.  Your points about
centralization are well taken, and I think the git approach used by
opam is a nice compromise: making it easy to have a centralized
source, without requiring it.

y

On Tue, Jul 23, 2013 at 3:49 PM, Pierre-Etienne Meunier
<pierreetienne.meunier@gmail.com> wrote:
>
> Em 23/07/2013, às 12:38, Yaron Minsky escreveu:
>
>> On Tue, Jul 23, 2013 at 11:32 AM, Pierre-Etienne Meunier  wrote:
>>> Hi,
>>>
>>> I'm only using my os' package manager at this time, so I am not part of the troll.
>>> But I find that Gerd made an interesting point, a few mails ago:
>>>
>>>> So far I've removed Core from GODI because it misses QA standards, and
>>>> have no plans to add it again.
>>
>> I was not planning on responding substantively to Gerd's point about
>> Core, but I don't want to let a false impression stand.
>
>
> I am sorry if I did not make myself clear: I was not trying to say anything about Core, nor Jane Street, nor you!
>
> My point is, I am scared by internet dictators, and, whether or not Core is a great library, I believe that no one, except maybe its authors, should have the right to remove it from this fantastic open public space that the internet is, and prevent its distribution. This is also what scares me when "github-friendly" is given as an argument for something.
>
> Pierre
>
>
> ps: IMHO, as one of its occasional users, Core *is* a great library. Thanks for sharing this colossal and amazing amount of work, Yaron.
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] GODI is shutting down

[Mike Lin]

[-- Attachment #1: Type: text/plain, Size: 3038 bytes --]

Gerd,
Thanks for all your work maintaining GODI over the years.
Mike



On Mon, Jul 22, 2013 at 2:21 PM, Gerd Stolpmann <info@gerd-stolpmann.de>wrote:

> Dear subscriber,
>
> Unfortunately, it is no longer possible for me to run the GODI
> distribution. GODI will not upgrade to OCaml 4.01 once it is out, and it
> will shut down the public service in the course of September 2013. The
> website, camlcity.org, will remain up, but with reduced content. Existing
> GODI installations can be continued to be used, but upgrades or bugfixes
> will not be available when GODI is off.
>
> Although there are still a lot of GODI users, it is unavoidable to shut
> GODI down due to lack of supporters, especially package developers. I was
> more or less alone in the past months, and my time contingent will not
> allow it to do the upgrade to OCaml 4.01 alone (when it is released).
>
> Also, there was a lot of noise about a competing packaging system for
> OCaml in the past weeks: OPAM. Apparently, it got a lot of attention both
> from individuals and from organizations. As I see it, the OCaml community
> is too small to support two systems, and so in some sense GODI is
> displaced by OPAM.
>
> The sad part is that OPAM is only clearly better in one point, namely in
> interacting with the community (via Github). In times where social
> networks are worth billions this is probably the striking point. It
> doesn't matter that OPAM lacks core functions like deleting all files when
> a package is removed, and that it lacks many other features GODI has. So
> there is some loss of functionality for the community (partly difficult to
> replace, like GODI's support for Windows).
>
> If somebody wants to take over GODI, please do so. The source code is
> still available as well as the package directories. Maybe it is sufficient
> to move the repository to a public place and to redesign the package
> release process to give GODI a restart.
>
> There is also another point that was driving me mad in the past weeks,
> namely missing respect from the OPAM guys. Given the fact that OPAM is
> only a thin layer around ocamlfind (and guess who wrote it), and given the
> fact that GODI was pioneering in many fields, I was expecting nicer
> wordings, and less dumb campaigning ("we have 400 packages, and you only
> 170"). OPAM is only harvesting what I seeded many years ago.
>
> Let's hope these guys get now some kicks into their asses, and are forced
> to add all the functionality to OPAM the OCaml community deserves.
>
> Hoorn (NL), the 22nd July 2013,
>
> Gerd Stolpmann
> --
> Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
> Creator of GODI and camlcity.org.
> Contact details:        http://www.camlcity.org/contact.html
> Company homepage:       http://www.gerd-stolpmann.de
>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

[-- Attachment #2: Type: text/html, Size: 4127 bytes --]

Re: [Caml-list] GODI is shutting down

[Francois Berenger]

On 07/23/2013 06:07 PM, Adrien Nader wrote:
> Hi,
 > [...]
> To be honest, I've never understood why opam was "started".

Contracted development, I guess.

 > I believe
> the time would have been much better spent improving godi.

Really?
So, when you start to dive in a large codebase, you think you have:
- full control
- deep understanding
- can do dramatic changes easily (like using state of the art
   dependency resolution technology)

I personally think it is good to start from scratch, sometimes.
In the long run, it can be a winning strategy.

> I've heard complaints against the UI and the package release process of
> godi.
> How can such complaints about the UI be a reason to start a new
> project?! Couldn't the package release process be improved gradually?

Some people also started another package manager before OPAM
went public, if you want to know. It was called odb.ml.

cf. https://github.com/thelema/odb

Maybe it was just a hack, but for some people it was already filling
the bill better than GODI.

F.

PS: I do like and use many stuffs released by Gerd: ocamlnet, ocamfind,
     etc.

Re: [Caml-list] GODI is shutting down

[Fabrice Le Fessant]

[-- Attachment #1: Type: text/plain, Size: 2588 bytes --]

On Wed, Jul 24, 2013 at 3:55 AM, Francois Berenger <berenger@riken.jp>wrote:

> On 07/23/2013 06:07 PM, Adrien Nader wrote:
>
>> Hi,
>>
> > [...]
>
>  To be honest, I've never understood why opam was "started".
>>
>
> Contracted development, I guess.


Yes, mostly. OCamlPro have had a contract with Jane Street since its
creation, on improving the OCaml environment, to the benefits of both Jane
Street and the whole OCaml community. The creation of a new package manager
was identified very early as a strategic element, to improve the usability
of OCaml, and increase its popularity. Thus, we started working on Opam, in
a collaboration between OCamlPro and INRIA (within the DORM european
project), with deep inputs from the Mancoosi team at University Paris
7/IRILL (working at improving Debian package management), and later joined
by OCamllabs as soon as it was created.

Of course, Opam would not have been the same without GODI: in its design,
Opam directly benefited from the experience of GODI, as we tried to keep
GODI's strengths and to find better alternatives to avoid its weaknesses.
We also studied some other package managers, for OCaml (odb, yypkg, etc.)
and for other languages/systems (Cabal, CPAN, ArchLinux, etc.). Finally, we
made sure we would be able to easily port GODI's packages to Opam, as the
number of available packages from the beginning is an important criteria
for adoption of a package management tool by end users.

Clearly, both GODI and Opam are technically challenging software, but they
are not focusing on solving the same technical challenges (as explained by
Thomas), although the functionalities they provide are globally similar.

As a side story, 15 years ago, I wrote one of the first open-source video
players for Divx files on Linux, in C++ with optimized MMX/SSE assembly
routines for zooming and so on. I was particularly proud of it, as it was a
domain in which I had little experience (but great interest ;-) ). I got a
few hundred users, when mplayer was released and all my users progressively
switched to it. Mplayer had support for some more video formats (but
nothing I could not implement), and some of my assembly routines were much
more efficient. Nonetheless, since then, I have been a happy user of
mplayer, and it is now much better technically than whatever I could have
done with my own player. This is typical of the software world, older
projects are superseded by new projects, not always better on all technical
grounds, but providing a different user experience or pushed by a larger
team of developers.

--Fabrice

[-- Attachment #2: Type: text/html, Size: 3302 bytes --]

Re: [Caml-list] GODI is shutting down

[Jun Furuse]

[-- Attachment #1: Type: text/plain, Size: 2981 bytes --]

Hi Fabrice,

Have you discussed your idea of OPAM with Gerd or GODI community before
starting the project? Was there no way of working together, to improve
GODI, or invite Gerd to OPAM development?

Best regards,
Jun
 On Jul 24, 2013 3:04 PM, "Fabrice Le Fessant" <Fabrice.Le_fessant@inria.fr>
wrote:

> On Wed, Jul 24, 2013 at 3:55 AM, Francois Berenger <berenger@riken.jp>wrote:
>
>> On 07/23/2013 06:07 PM, Adrien Nader wrote:
>>
>>> Hi,
>>>
>> > [...]
>>
>>  To be honest, I've never understood why opam was "started".
>>>
>>
>> Contracted development, I guess.
>
>
> Yes, mostly. OCamlPro have had a contract with Jane Street since its
> creation, on improving the OCaml environment, to the benefits of both Jane
> Street and the whole OCaml community. The creation of a new package manager
> was identified very early as a strategic element, to improve the usability
> of OCaml, and increase its popularity. Thus, we started working on Opam, in
> a collaboration between OCamlPro and INRIA (within the DORM european
> project), with deep inputs from the Mancoosi team at University Paris
> 7/IRILL (working at improving Debian package management), and later joined
> by OCamllabs as soon as it was created.
>
> Of course, Opam would not have been the same without GODI: in its design,
> Opam directly benefited from the experience of GODI, as we tried to keep
> GODI's strengths and to find better alternatives to avoid its weaknesses.
> We also studied some other package managers, for OCaml (odb, yypkg, etc.)
> and for other languages/systems (Cabal, CPAN, ArchLinux, etc.). Finally, we
> made sure we would be able to easily port GODI's packages to Opam, as the
> number of available packages from the beginning is an important criteria
> for adoption of a package management tool by end users.
>
> Clearly, both GODI and Opam are technically challenging software, but they
> are not focusing on solving the same technical challenges (as explained by
> Thomas), although the functionalities they provide are globally similar.
>
> As a side story, 15 years ago, I wrote one of the first open-source video
> players for Divx files on Linux, in C++ with optimized MMX/SSE assembly
> routines for zooming and so on. I was particularly proud of it, as it was a
> domain in which I had little experience (but great interest ;-) ). I got a
> few hundred users, when mplayer was released and all my users progressively
> switched to it. Mplayer had support for some more video formats (but
> nothing I could not implement), and some of my assembly routines were much
> more efficient. Nonetheless, since then, I have been a happy user of
> mplayer, and it is now much better technically than whatever I could have
> done with my own player. This is typical of the software world, older
> projects are superseded by new projects, not always better on all technical
> grounds, but providing a different user experience or pushed by a larger
> team of developers.
>
> --Fabrice
>
>

[-- Attachment #2: Type: text/html, Size: 3873 bytes --]

Re: [Caml-list] GODI is shutting down

[Mihamina Rakotomandimby]

[-- Attachment #1: Type: text/plain, Size: 192 bytes --]

On 2013-07-23 16:31, Marek Kubica wrote:
> It is just that OPAM seemed to be a good deal more active.
                         ^^^^^
Well, you understood Gerd's point.
Just my POV.

-- 
RMA.


[-- Attachment #2: Type: text/html, Size: 804 bytes --]

[Caml-list] Re: GODI is shutting down

[Andreas Hauptmann]

Hi,

On Tue, 23 Jul 2013 10:51:22 +0900
Francois Berenger <berenger@riken.jp> wrote:
> I wonder how many users of GODI on windows there are.
> 
> This is clearly the big plus over OPAM for the moment, I feel.

My windows fork ( http://wodi.forge.ocamlcore.org/ ) is independent
from Gerd's servers. It won't be affected by the shutdown in September.
If there will be a 4.01 release soon, I intend to publish updated
package informations and binary builds for Windows.

There are not many users. Therefore, I was too lazy to add an own
package release infrastructure (via github or similar). Up to now, the
packages were mainly auto-generated or synchronized from the mainline
repository.


cu,
 Andreas

Re: [Caml-list] GODI is shutting down

[Daniel Bünzli]

Le mercredi, 24 juillet 2013 à 09:42, Jun Furuse a écrit :
> Have you discussed your idea of OPAM with Gerd or GODI community before starting the project? Was there no way of working together, to improve GODI, or invite Gerd to OPAM development?

I'm sorry but these questions are pointless.  

Since when do we have to ask for permissions to start new projects ? Since when do we have to cooperate with other people making a similar system if we have divergent vision of what we want ? I think that Francois Berenger made a few good points on why you may want to start a new project.  

Sure, what happens most of the time when an individual says I'm not happy with your system I'm going to implement my own, he fails badly because he doesn't actually understand the work/complexity involved. People tend to see that as a wasted time, wasted time for the "community" bla bla, but I don't, that's the way you learn.

In that case it wasn't a failure and, at least for me, I'm now given a system I actually enjoy working with from a *technical* and *usability* perspective both as a package user and as a package developer. It has nothing to do with a supposed marketing campaign that I never saw. I tried the damn thing, liked it, considered the efforts that are being put in it and decided to use it.

I find it strange that for quite sometime we had other individuals working on a similar concurrent project (oasis-db) and nobody every screamed at them. That project never delivered and now that we have people actually delivering we should scream at them ?

I tried godi a very long time ago and was not happy about the centralized workflow, maybe that changed I don't know. I then used odb for sometime because it was working for the simple things I did and it was easy to hack with or provide your own package sources but I still saw it as a quick hack --- albeit a very useful one for the time it served me, thanks to Edgar Friendly.

Now I use opam and while there are still a few things that needs to be ironed out for my development practice. I think that for myself it will eventually be of great help to manage the growing collection of packages I provide and streamline my release process and admin time.  

Best,

Daniel

AW: [Caml-list] GODI is shutting down

[Gerd Stolpmann]

Let me answer to this message, because it is again fact-based, and not  
full of unproven or poor-quality claims.

I hope the reader can now understand that OPAM is set up as competing  
project from the beginning. Basically, three organizations joined to  
create a new software. The discussion about the design wasn't public. I  
was excluded from the beginning - and don't understand why, since I  
made an offer for cooperation when I first heard from Ocamlpro's plans  
(not about OPAM in particular, but other stuff, but I suggested to  
cooperate in package manager questions). I never got a response to my  
initiative. Instead, there was later questionable behavior from  
Ocamlpro, namely that they copied (at least parts of) GODI packages,  
without having asked for permission. (In particular, there was evidence  
that they copied the package description texts.) This might be legal,  
but the reader will understand that this is unfriendly, and  
unneccessarily aggressive. At that point it was clear that they are  
trying to push GODI aside, even with questionable means.

In the past days I made the experience that OPAM advocates never answer  
to any of my objections. When I said "GODI has feature xy, why doesn't  
OPAM has this too?" the only response is something like "OPAM has this  
great Github-based workflow" (the most harmless variant). Discussion  
impossible.

Fabrice claims that OPAM has GODI's strengths and tries to avoid its  
weaknesses. I'd say this is just a personal view, because there was  
never a public discussion (and Fabrice would probably be surprised what  
I consider as GODI's strengths). I know very well where GODI's  
weaknesses are, but I'm sure a public discussion would reveal that  
there are a number of useful features that are currently missing in  
OPAM, and where "we use Github" is no excuse. It would be very helpful  
if the OPAM developers would a bit more self-critical, and did not  
continuously claim that their software is superior (well, you know  
Github, and dose3).

Let me only add one fundamental point where I think OPAM just takes  
just one direction, and that it is worth a discussion never done. One  
of the principles seems to be to make it as easy and comfortable as  
possible to package up libraries. At the first glance a good thing, but  
if you look deeper you'll see that this principle collides with other  
ideas, and at least a modification of the principle is worth a thought.  
There is the QA question, at least for the main repository (NB. as the  
bad word of "dictator" was mentioned - yes, for the main repository  
there is a dictator, both in OPAM and GODI, but belive me there is  
enough anti-power). There are features in package management that are a  
hassle for the packager, like clean deinstallation, or how well a  
library is integrated into the OS. So far I've the impression OPAM just  
ignores this, in the hope to attract as many packagers as possible. It  
would be a bad deal if packager's interests always won over user's  
interests.

A final remark to Fabrice's side story. Of course, it is normal that  
software is suddenly no popular anymore, and users switch to a  
different product. The user decides, after all, and developers  
shouldn't take this personally. The software market is a jungle.  
Nevertheless, this doesn't mean we shouldn't try to be civilized.  
Often, there is an alternative (e.g. cooperation), and most often the  
civilized way leads to better results (e.g. imagine mplayer would use  
your assembly routines).

Gerd

Am 24.07.2013 09:03:46 schrieb(en) Fabrice Le Fessant:
> On Wed, Jul 24, 2013 at 3:55 AM, Francois Berenger  
> <berenger@riken.jp>wrote:
> 
> > On 07/23/2013 06:07 PM, Adrien Nader wrote:
> >
> >> Hi,
> >>
> > > [...]
> >
> >  To be honest, I've never understood why opam was "started".
> >>
> >
> > Contracted development, I guess.
> 
> 
> Yes, mostly. OCamlPro have had a contract with Jane Street since its
> creation, on improving the OCaml environment, to the benefits of both  
> Jane
> Street and the whole OCaml community. The creation of a new package  
> manager
> was identified very early as a strategic element, to improve the  
> usability
> of OCaml, and increase its popularity. Thus, we started working on  
> Opam, in
> a collaboration between OCamlPro and INRIA (within the DORM european
> project), with deep inputs from the Mancoosi team at University Paris
> 7/IRILL (working at improving Debian package management), and later  
> joined
> by OCamllabs as soon as it was created.
> 
> Of course, Opam would not have been the same without GODI: in its  
> design,
> Opam directly benefited from the experience of GODI, as we tried to  
> keep
> GODI's strengths and to find better alternatives to avoid its  
> weaknesses.
> We also studied some other package managers, for OCaml (odb, yypkg,  
> etc.)
> and for other languages/systems (Cabal, CPAN, ArchLinux, etc.).  
> Finally, we
> made sure we would be able to easily port GODI's packages to Opam, as  
> the
> number of available packages from the beginning is an important  
> criteria
> for adoption of a package management tool by end users.
> 
> Clearly, both GODI and Opam are technically challenging software, but  
> they
> are not focusing on solving the same technical challenges (as  
> explained by
> Thomas), although the functionalities they provide are globally  
> similar.
> 
> As a side story, 15 years ago, I wrote one of the first open-source  
> video
> players for Divx files on Linux, in C++ with optimized MMX/SSE  
> assembly
> routines for zooming and so on. I was particularly proud of it, as it  
> was a
> domain in which I had little experience (but great interest ;-) ). I  
> got a
> few hundred users, when mplayer was released and all my users  
> progressively
> switched to it. Mplayer had support for some more video formats (but
> nothing I could not implement), and some of my assembly routines were  
> much
> more efficient. Nonetheless, since then, I have been a happy user of
> mplayer, and it is now much better technically than whatever I could  
> have
> done with my own player. This is typical of the software world, older
> projects are superseded by new projects, not always better on all  
> technical
> grounds, but providing a different user experience or pushed by a  
> larger
> team of developers.
> 
> --Fabrice
> 
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs



-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------

Re: AW: [Caml-list] GODI is shutting down

[Thomas Gazagnaire]

> In the past days I made the experience that OPAM advocates never answer to any of my objections. When I said "GODI has feature xy, why doesn't OPAM has this too?" the only response is something like "OPAM has this great Github-based workflow" (the most harmless variant). Discussion impossible.

Let's try to be constructive then. I've tried to look for all your recent objections, let me known if I have missed some.

1. " Instead, there was later questionable behavior from Ocamlpro, namely that they copied (at least parts of) GODI packages, without having asked for permission. (In particular, there was evidence that they copied the package description texts.) This might be legal, but the reader will understand that this is unfriendly, and unneccessarily aggressive. At that point it was clear that they are trying to push GODI aside, even with questionable means."

Some of the initial package descriptions were indeed copied from the description files of the corresponding GODI packages (which have been often itself copied from the author website). I was not aware of this when the descriptions got integrated in the repository, and I've removed them as soon as the issue has been raised on the GODI mailing list [1, 8 oct 2012] and [2, 8 oct 2012]. To avoid such issue in the future we are planning to clarify the licensing scheme on all the metadata published on the "official repository" (we would like to make it public domain if possible, but it's maybe too late).

[1] https://godirepo.camlcity.org/pipermail/godi-list/2012-October/003517.html
[2] https://godirepo.camlcity.org/pipermail/godi-list/2012-October/003521.html

2. "It doesn't matter that OPAM lacks core functions like deleting all files when a package is removed"

I was very surprised when I've read that (and I though it was just an unrelated rant), but as you seem to insist I guess you are serious. Where did you read that ? Of course, OPAM deletes the files it has installed!

- It removes $prefix/lib/<name>, $prefix/doc/<name> (and the other usual paths associated to a package, but keeps $share/<name> for future reinstallations) and it calls whatever removal scripts the user has registered (which is 'ocamlfind remove <name>' for most of the packaged libaries -- which is still useful when you have C stubs to remove in $prefix/stublibs), and;

- It also removes the files specified in an optional `<name>.install` file located at the root of the directory (which is useful when you have binaries). We have some experimental features to scan the filesystem before and after the installation, and update the corresponding `<name>.install` file with the difference but it's quite slow (because of the filesystem scanning) and this does not work when the number of jobs is greater than 1. But I would prefer to let that task to the packagers (or to their build-system, see for instance that nice omake extension[3])

[3] https://github.com/smondet/dircmp/blob/master/OMakeroot#L175

3. "There is the QA question, at least for the main repository"

This is on-going work. The first objective was to gather as much packages as possible, and to make people start to rely on other people packages instead of restarting each projects from scratch. Now we are starting to focus on the QA side, as Anil already have already told you in a previous email, as we slowly replace our old private Jenkins setup by a more advanced, fully written in OCaml -- but still experimental -- testing platform. Look at the the 4.00.1 on linux results they are the more meaningful for now on, more than 90% of the packages are working fine (and yes, that's definitely needs to be improved, but I would say that's not so bad).

https://ocaml-www3.ocamllabs.cl.cam.ac.uk/github/OCamlPro/opam-repository

Then, as discussed on [4], we plan to select the package with the right level of QA to be part of the upcoming "OCaml Platform". The platform itself will either be a separate repository, or a separate branch or simply a virtual package in the main repository -- as explained in [4] (and on other threads of the platform list) the contents of the platform will be driven by QA criteria.

[4] http://lists.ocaml.org/pipermail/platform/2013-February/000001.html

4. "how well a library is integrated into the OS"

Currently, with a minimal support. We are working on improving that. Look at the `depext` field in the sqlite3-ocaml package[5], that you can query using the command-line:

$ opam install sqlite3-ocaml -e debian,wheezy,amd64
libsqlite3-dev

The goal is to use that metadata to "drive" the installation/checks of external dependencies, depending on the user host and OS. The short-term goal is to display nice error messages though.

[5] https://github.com/OCamlPro/opam-repository/blob/master/packages/sqlite3-ocaml.2.0.4/opam

5. "lack of windows support and binary packages"

OPAM supports for Windows is not so good at the moment indeed. However, things are getting better as we are improving the support for windows (we have a private version which compiles and run OK on cygwin for basic packages) and we have an experimental binary support that we are still prototyping [6]. GODI and WODI are clearly in advance in this domain (but I recall that WODI is a relatively new project).

[6]. https://github.com/venator/opam/tree/binary

6. "No, all package managers should unite in this point, and only accept
packages with oasis support. (Btw, that's homework for me.) Just do it the
same way as we did when requiring findlib."

I disagree, people should be free to use whatever system they want.

7. About the generation of OPAM files from _oasis

I am not totally convinced by all the arguments I seen so far. Making a new release because a package constraint is wrong seems not to be a good idea to me. A large proportion of pull request in the the package metadata's repository is about fixing such dependency constraints (which are discovered by our testing tools). I don't have a yet a good story to bring that information back into the package, but I'd say that I'm not very fond of mixing everything together: let's keep the dependency constraints part of the package system and the build instruction part of the build system -- that's much simpler that way.

8. "You are a victim of OPAM's campaign."

Sorry, I have no facts to answer here.


Best,
Thomas

Re: AW: [Caml-list] GODI is shutting down

[Markus Mottl]

On Wed, Jul 24, 2013 at 10:44 AM, Thomas Gazagnaire <thomas@ocamlpro.com> wrote:
> 6. "No, all package managers should unite in this point, and only accept
> packages with oasis support. (Btw, that's homework for me.) Just do it the
> same way as we did when requiring findlib."
>
> I disagree, people should be free to use whatever system they want.

Let me add here that avoiding limitations or obstacles does not
necessarily create more freedom.  I, as a package developer, want to
have the freedom of not having to worry about annoyances in the
packaging process that are only there so that unlikely alternatives
can be supported.  Lets keep the simple (= standard) things simple
while keeping the non-standard things possible, but I'd prefer
simplicity if there is a conflict.

Oasis may are may not become the standard OCaml package specification
tool, but it is widely used so I think it's fair if it is better
supported by OPAM than alternatives.

> 7. About the generation of OPAM files from _oasis
>
> I am not totally convinced by all the arguments I seen so far. Making a new release because a package constraint is wrong seems not to be a good idea to me. A large proportion of pull request in the the package metadata's repository is about fixing such dependency constraints (which are discovered by our testing tools). I don't have a yet a good story to bring that information back into the package, but I'd say that I'm not very fond of mixing everything together: let's keep the dependency constraints part of the package system and the build instruction part of the build system -- that's much simpler that way.

It's funny that you say that, because this collides with your opinion
on 6).  This would mean forcing people to use OPAM to specify package
version dependencies instead of leaving this information in their
packages.  That's why I think that requiring (or at least
well-supporting) something like Oasis package specifications as
standard adds freedom for package developers.

My guess is that you are not totally happy with Oasis at this point so
this may be a topic to address.  Being an Oasis user, I would say that
the Oasis tools may not be as mature yet as one would hope, but the
Cabal-like package specification syntax seems decent enough to be
supported by OPAM.  It really shouldn't be that hard to automate the
process of updating OPAM metadata (including version constraints)
based on changes in Oasis package specifications.

Regards,
Markus

--
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com

Re: AW: [Caml-list] GODI is shutting down

[David Sheets]

On Wed, Jul 24, 2013 at 5:18 PM, Matej Kosik
<5764c029b688c1c0d24a2e97cd764f@gmail.com> wrote:
> On 24/07/13 15:44, Thomas Gazagnaire wrote:
>>> In the past days I made the experience that OPAM advocates never answer to any of my objections. When I said "GODI has feature xy, why doesn't OPAM has this too?" the only response is something like "OPAM has this great Github-based workflow" (the most harmless variant). Discussion impossible.
>>
>> Let's try to be constructive then. I've tried to look for all your recent objections, let me known if I have missed some.
>>
>> 1. " Instead, there was later questionable behavior from Ocamlpro, namely that they copied (at least parts of) GODI packages, without having asked for permission. (In particular, there was evidence that they copied the package description texts.) This might be legal, but the reader will understand that this is unfriendly, and unneccessarily aggressive. At that point it was clear that they are trying to push GODI aside, even with questionable means."
>>
>> Some of the initial package descriptions were indeed copied from the description files of the corresponding GODI packages (which have been often itself copied from the author website). I was not aware of this when the descriptions got integrated in the repository, and I've removed them as soon as the issue has been raised on the GODI mailing list [1, 8 oct 2012] and [2, 8 oct 2012]. To avoid such issue in the future we are planning to clarify the licensing scheme on all the metadata published on the "official repository" (we would like to make it public domain if possible, but it's maybe too late).
>>
>> [1] https://godirepo.camlcity.org/pipermail/godi-list/2012-October/003517.html
>> [2] https://godirepo.camlcity.org/pipermail/godi-list/2012-October/003521.html
>>
>> 2. "It doesn't matter that OPAM lacks core functions like deleting all files when a package is removed"
>>
>> I was very surprised when I've read that (and I though it was just an unrelated rant), but as you seem to insist I guess you are serious. Where did you read that ? Of course, OPAM deletes the files it has installed!
>>
>> - It removes $prefix/lib/<name>, $prefix/doc/<name> (and the other usual paths associated to a package, but keeps $share/<name> for future reinstallations) and it calls whatever removal scripts the user has registered (which is 'ocamlfind remove <name>' for most of the packaged libaries -- which is still useful when you have C stubs to remove in $prefix/stublibs), and;
>>
>> - It also removes the files specified in an optional `<name>.install` file located at the root of the directory (which is useful when you have binaries). We have some experimental features to scan the filesystem before and after the installation, and update the corresponding `<name>.install` file with the difference but it's quite slow (because of the filesystem scanning) and this does not work when the number of jobs is greater than 1. But I would prefer to let that task to the packagers (or to their build-system, see for instance that nice omake extension[3])
>>
>> [3] https://github.com/smondet/dircmp/blob/master/OMakeroot#L175
>>
>> 3. "There is the QA question, at least for the main repository"
>>
>> This is on-going work. The first objective was to gather as much packages as possible, and to make people start to rely on other people packages instead of restarting each projects from scratch. Now we are starting to focus on the QA side, as Anil already have already told you in a previous email, as we slowly replace our old private Jenkins setup by a more advanced, fully written in OCaml -- but still experimental -- testing platform. Look at the the 4.00.1 on linux results they are the more meaningful for now on, more than 90% of the packages are working fine (and yes, that's definitely needs to be improved, but I would say that's not so bad).
>>
>> https://ocaml-www3.ocamllabs.cl.cam.ac.uk/github/OCamlPro/opam-repository
>>
>> Then, as discussed on [4], we plan to select the package with the right level of QA to be part of the upcoming "OCaml Platform". The platform itself will either be a separate repository, or a separate branch or simply a virtual package in the main repository -- as explained in [4] (and on other threads of the platform list) the contents of the platform will be driven by QA criteria.
>>
>> [4] http://lists.ocaml.org/pipermail/platform/2013-February/000001.html
>>
>> 4. "how well a library is integrated into the OS"
>>
>> Currently, with a minimal support. We are working on improving that. Look at the `depext` field in the sqlite3-ocaml package[5], that you can query using the command-line:
>>
>> $ opam install sqlite3-ocaml -e debian,wheezy,amd64
>> libsqlite3-dev
>>
>> The goal is to use that metadata to "drive" the installation/checks of external dependencies, depending on the user host and OS. The short-term goal is to display nice error messages though.
>>
>> [5] https://github.com/OCamlPro/opam-repository/blob/master/packages/sqlite3-ocaml.2.0.4/opam
>>
>> 5. "lack of windows support and binary packages"
>>
>> OPAM supports for Windows is not so good at the moment indeed. However, things are getting better as we are improving the support for windows (we have a private version which compiles and run OK on cygwin for basic packages) and we have an experimental binary support that we are still prototyping [6]. GODI and WODI are clearly in advance in this domain (but I recall that WODI is a relatively new project).
>>
>> [6]. https://github.com/venator/opam/tree/binary
>>
>> 6. "No, all package managers should unite in this point, and only accept
>> packages with oasis support. (Btw, that's homework for me.) Just do it the
>> same way as we did when requiring findlib."
>>
>> I disagree, people should be free to use whatever system they want.
>
> The above argument, in general, is invalid.

[citation needed]

>>
>> 7. About the generation of OPAM files from _oasis
>>
>> I am not totally convinced by all the arguments I seen so far. Making a new release because a package constraint is wrong seems not to be a good idea to me. A large proportion of pull request in the the package metadata's repository is about fixing such dependency constraints (which are discovered by our testing tools). I don't have a yet a good story to bring that information back into the package, but I'd say that I'm not very fond of mixing everything together: let's keep the dependency constraints part of the package system and the build instruction part of the build system -- that's much simpler that way.
>>
>> 8. "You are a victim of OPAM's campaign."
>>
>> Sorry, I have no facts to answer here.
>>
>>
>> Best,
>> Thomas
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: AW: [Caml-list] GODI is shutting down

[Matej Kosik]

On 24/07/13 15:44, Thomas Gazagnaire wrote:
>> In the past days I made the experience that OPAM advocates never answer to any of my objections. When I said "GODI has feature xy, why doesn't OPAM has this too?" the only response is something like "OPAM has this great Github-based workflow" (the most harmless variant). Discussion impossible.
> 
> Let's try to be constructive then. I've tried to look for all your recent objections, let me known if I have missed some.
> 
> 1. " Instead, there was later questionable behavior from Ocamlpro, namely that they copied (at least parts of) GODI packages, without having asked for permission. (In particular, there was evidence that they copied the package description texts.) This might be legal, but the reader will understand that this is unfriendly, and unneccessarily aggressive. At that point it was clear that they are trying to push GODI aside, even with questionable means."
> 
> Some of the initial package descriptions were indeed copied from the description files of the corresponding GODI packages (which have been often itself copied from the author website). I was not aware of this when the descriptions got integrated in the repository, and I've removed them as soon as the issue has been raised on the GODI mailing list [1, 8 oct 2012] and [2, 8 oct 2012]. To avoid such issue in the future we are planning to clarify the licensing scheme on all the metadata published on the "official repository" (we would like to make it public domain if possible, but it's maybe too late).
> 
> [1] https://godirepo.camlcity.org/pipermail/godi-list/2012-October/003517.html
> [2] https://godirepo.camlcity.org/pipermail/godi-list/2012-October/003521.html
> 
> 2. "It doesn't matter that OPAM lacks core functions like deleting all files when a package is removed"
> 
> I was very surprised when I've read that (and I though it was just an unrelated rant), but as you seem to insist I guess you are serious. Where did you read that ? Of course, OPAM deletes the files it has installed!
> 
> - It removes $prefix/lib/<name>, $prefix/doc/<name> (and the other usual paths associated to a package, but keeps $share/<name> for future reinstallations) and it calls whatever removal scripts the user has registered (which is 'ocamlfind remove <name>' for most of the packaged libaries -- which is still useful when you have C stubs to remove in $prefix/stublibs), and;
> 
> - It also removes the files specified in an optional `<name>.install` file located at the root of the directory (which is useful when you have binaries). We have some experimental features to scan the filesystem before and after the installation, and update the corresponding `<name>.install` file with the difference but it's quite slow (because of the filesystem scanning) and this does not work when the number of jobs is greater than 1. But I would prefer to let that task to the packagers (or to their build-system, see for instance that nice omake extension[3])
> 
> [3] https://github.com/smondet/dircmp/blob/master/OMakeroot#L175
> 
> 3. "There is the QA question, at least for the main repository"
> 
> This is on-going work. The first objective was to gather as much packages as possible, and to make people start to rely on other people packages instead of restarting each projects from scratch. Now we are starting to focus on the QA side, as Anil already have already told you in a previous email, as we slowly replace our old private Jenkins setup by a more advanced, fully written in OCaml -- but still experimental -- testing platform. Look at the the 4.00.1 on linux results they are the more meaningful for now on, more than 90% of the packages are working fine (and yes, that's definitely needs to be improved, but I would say that's not so bad).
> 
> https://ocaml-www3.ocamllabs.cl.cam.ac.uk/github/OCamlPro/opam-repository
> 
> Then, as discussed on [4], we plan to select the package with the right level of QA to be part of the upcoming "OCaml Platform". The platform itself will either be a separate repository, or a separate branch or simply a virtual package in the main repository -- as explained in [4] (and on other threads of the platform list) the contents of the platform will be driven by QA criteria.
> 
> [4] http://lists.ocaml.org/pipermail/platform/2013-February/000001.html
> 
> 4. "how well a library is integrated into the OS"
> 
> Currently, with a minimal support. We are working on improving that. Look at the `depext` field in the sqlite3-ocaml package[5], that you can query using the command-line:
> 
> $ opam install sqlite3-ocaml -e debian,wheezy,amd64
> libsqlite3-dev
> 
> The goal is to use that metadata to "drive" the installation/checks of external dependencies, depending on the user host and OS. The short-term goal is to display nice error messages though.
> 
> [5] https://github.com/OCamlPro/opam-repository/blob/master/packages/sqlite3-ocaml.2.0.4/opam
> 
> 5. "lack of windows support and binary packages"
> 
> OPAM supports for Windows is not so good at the moment indeed. However, things are getting better as we are improving the support for windows (we have a private version which compiles and run OK on cygwin for basic packages) and we have an experimental binary support that we are still prototyping [6]. GODI and WODI are clearly in advance in this domain (but I recall that WODI is a relatively new project).
> 
> [6]. https://github.com/venator/opam/tree/binary
> 
> 6. "No, all package managers should unite in this point, and only accept
> packages with oasis support. (Btw, that's homework for me.) Just do it the
> same way as we did when requiring findlib."
> 
> I disagree, people should be free to use whatever system they want.

The above argument, in general, is invalid.

> 
> 7. About the generation of OPAM files from _oasis
> 
> I am not totally convinced by all the arguments I seen so far. Making a new release because a package constraint is wrong seems not to be a good idea to me. A large proportion of pull request in the the package metadata's repository is about fixing such dependency constraints (which are discovered by our testing tools). I don't have a yet a good story to bring that information back into the package, but I'd say that I'm not very fond of mixing everything together: let's keep the dependency constraints part of the package system and the build instruction part of the build system -- that's much simpler that way.
> 
> 8. "You are a victim of OPAM's campaign."
> 
> Sorry, I have no facts to answer here.
> 
> 
> Best,
> Thomas

Re: AW: [Caml-list] GODI is shutting down

[Thomas Gazagnaire]

> Oasis may are may not become the standard OCaml package specification
> tool, but it is widely used so I think it's fair if it is better
> supported by OPAM than alternatives.

Right, this is indeed a feature requested by many people, than I can hear. This is tracked and discussed in [1] and I'm fine to try to implement the proposed solution in 1.1 (I've just updated the issue's release target).

[1] https://github.com/OCamlPro/opam/issues/156

> It really shouldn't be that hard to automate the
> process of updating OPAM metadata (including version constraints)
> based on changes in Oasis package specifications.

A substantial amount of work has already been done by Christophe Troestler to automate the conversion of _oasis  into OPAM files:

    $ opam install oasis2opam

I'm sure this can be completed to automate the pull-request machinery as well.

However, the main problem (the one I wanted to point out) still remains: every time you want to fix a constraint you have to change your _oasis, modify the package tarball, and make a new release -- which means you can never fix constraints on already released packages. But I guess in this case, it's fine if the _oasis and OPAM files are not in sync.

[2] https://github.com/ocaml/oasis2opam

--
Thomas

Re: AW: [Caml-list] GODI is shutting down

[Gabriel Scherer]

> However, the main problem (the one I wanted to point out) still remains: every time you want to fix a constraint you have to change your _oasis, modify the package tarball, and make a new release -- which means you can never fix constraints on already released packages. But I guess in this case, it's fine if the _oasis and OPAM files are not in sync.

You could include a patch against the _oasis file in the OPAM (or
GODI) metadata.

On Wed, Jul 24, 2013 at 6:25 PM, Thomas Gazagnaire <thomas@ocamlpro.com> wrote:
>> Oasis may are may not become the standard OCaml package specification
>> tool, but it is widely used so I think it's fair if it is better
>> supported by OPAM than alternatives.
>
> Right, this is indeed a feature requested by many people, than I can hear. This is tracked and discussed in [1] and I'm fine to try to implement the proposed solution in 1.1 (I've just updated the issue's release target).
>
> [1] https://github.com/OCamlPro/opam/issues/156
>
>> It really shouldn't be that hard to automate the
>> process of updating OPAM metadata (including version constraints)
>> based on changes in Oasis package specifications.
>
> A substantial amount of work has already been done by Christophe Troestler to automate the conversion of _oasis  into OPAM files:
>
>     $ opam install oasis2opam
>
> I'm sure this can be completed to automate the pull-request machinery as well.
>
> However, the main problem (the one I wanted to point out) still remains: every time you want to fix a constraint you have to change your _oasis, modify the package tarball, and make a new release -- which means you can never fix constraints on already released packages. But I guess in this case, it's fine if the _oasis and OPAM files are not in sync.
>
> [2] https://github.com/ocaml/oasis2opam
>
> --
> Thomas
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: AW: [Caml-list] GODI is shutting down

[Markus Mottl]

On Wed, Jul 24, 2013 at 12:25 PM, Thomas Gazagnaire <thomas@ocamlpro.com> wrote:
> However, the main problem (the one I wanted to point out) still remains: every time you want to fix a constraint you have to change your _oasis, modify the package tarball, and make a new release -- which means you can never fix constraints on already released packages. But I guess in this case, it's fine if the _oasis and OPAM files are not in sync.

The way I see it is that a package that contains incorrect dependency
information, even if it's just a version thing, has a bug and should
be fixed.  I agree this is annoying to package developers, but fixing
bugs is always annoying ;-)

I don't see a conflict with OPAM allowing you to fix dependency
information just within OPAM.  Maybe there should be some special
syntax to identify overrides to constraints for certain package
versions so that corrected version constraints will eventually be
picked up from fixed packages again.  In the long run my preferred
solution is to fix the cause of a problem, not to use band aids to
cover up symptoms.

Regards,
Markus

--
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com

Re: [Caml-list] GODI is shutting down

[Anil Madhavapeddy]

On 24 Jul 2013, at 08:58, Markus Mottl <markus.mottl@gmail.com> wrote:

> My guess is that you are not totally happy with Oasis at this point so
> this may be a topic to address.  Being an Oasis user, I would say that
> the Oasis tools may not be as mature yet as one would hope, but the
> Cabal-like package specification syntax seems decent enough to be
> supported by OPAM.  It really shouldn't be that hard to automate the
> process of updating OPAM metadata (including version constraints)
> based on changes in Oasis package specifications.

I think that's fair, but the devil is in the details. The core of OASIS
is sound, but is hampered by its need to build on top of ocamlfind
and ocamlbuild, and the ensuing impedance mismatches that follow (e.g.
manually editing _tags files, and trying to debug which tool a failure
happens in is an advanced operation). OPAM packages represent an output
collection of (several optionally built) ocamlfind packages, and 
OASIS sits awkwardly in between these two right now, with ocamlbuild
crashing the party and getting drunk in the kitchen.

OASIS is structured very well to help get around this situation though.
It has a library that takes care of parsing the _oasis file, and then a
plugin generator that currently uses ocamlbuild to drive this specification
into concrete outputs.  Meanwhile, the OCaml compiler has been exposing
more of its innards via compiler-libs. Leo and I are taking advantage of
this by trying to folding all of these layers into a new "packaging-friendly"
OCaml driver that statically links:

- all the camlp4 libraries required for a project, and directly calls it
  instead of going via -pp (avoiding a fork/marshal).
- links to OPAM/findlib to find dependencies directly, avoiding a shell
  out to ocamlfind.
- can publish a static schedule of its activities to help Makefiles
  figure out what actually got built (similar to the recent ocamlbuild
  request).

The idea is to have a "ocamlmkcompiler" that builds a binary for-purpose,
as "ocamlmktop" does today for toplevels.  It shouldn't even need any
patches to the official compiler, thanks to compiler-libs.  More to the
point, it should make it far easier to build a camlp4-aware js_of_ocaml
toplevel, to fit into the experiments that OCamlPro is doing with web
IDE interfaces.

Anyway, these are still experiments and quite likely to run into unexpected
issues as we build them, and we'll have to consider all the alternatives
before folding them all into OPAM.

Right now, OPAM dictates the *minimal* possible set of requirements for
a package since it just uses shell commands.  It should be possible to
specialise this for common cases such as a OASIS library, but it does
require some careful thought first.  Luckily, we can use OPAM itself to
conduct such experiments [1].

All such discussions and experiments are very welcome on the mailing
list (http://lists.ocaml.org/listinfo/opam-devel) to help establish
everyone's use cases.  Daniel Buenzli's input has been particularly
interesting to date, as his libraries adopt a very minimal shell-script
driven approach that doesn't use OASIS, but still distributes META files.
That's just one example of someone that doesn't use OASIS directly.
The Core OASIS files are also a bit special due to their configuration
requirements.  Xen/Mirage is also quite demanding in terms of the 
compilation environment.

Here's my goal: I want to have near instant-rebuilds of Mirage packages.
I know it's possible to pull this off with the existing compiler tools,
since the monolithic Mirage tree (with lovingly crafted ocamlbuild rules
and judicious use of native code compiler binaries) could rebuild the
entire webserver in ~10s in parallel.  Nowadays, with OPAM, ocamlbuild,
ocamlfind and camlp4, it takes around 2-3 minutes, which just isn't
good enough.

-anil

[1] http://opam.ocamlpro.com/pkg/debug-camlp4-log.base.html: I hacked
up a debug camlp4 to trace all the uses of camlp4 in the OPAM repository,
during the wg-camlp4 discussion on the topic of where to go next with it.
This just required doing a mass OPAM install, drinking some coffee, and
reading the resulting trace logs. 

Re: [Caml-list] GODI is shutting down

[Anil Madhavapeddy]

On 24 Jul 2013, at 09:36, Gabriel Scherer <gabriel.scherer@gmail.com> wrote:

>> However, the main problem (the one I wanted to point out) still remains: every time you want to fix a constraint you have to change your _oasis, modify the package tarball, and make a new release -- which means you can never fix constraints on already released packages. But I guess in this case, it's fine if the _oasis and OPAM files are not in sync.
> 
> You could include a patch against the _oasis file in the OPAM (or
> GODI) metadata.

That's not enough.  You also need a build-time requirement to regenerate
the OASIS autogenerated files by calling 'oasis setup', which increases
the total build time significantly.

Some projects (such as Mirage) also use a forked OASIS to add some
specialised features in the generated output, and so need special rules
to regenerate them.

-anil

Re: AW: [Caml-list] GODI is shutting down

[Matej Kosik]

On 24/07/13 17:17, David Sheets wrote:
> On Wed, Jul 24, 2013 at 5:18 PM, Matej Kosik
> <5764c029b688c1c0d24a2e97cd764f@gmail.com> wrote:
>> On 24/07/13 15:44, Thomas Gazagnaire wrote:
>>>
>>> 6. "No, all package managers should unite in this point, and only accept
>>> packages with oasis support. (Btw, that's homework for me.) Just do it the
>>> same way as we did when requiring findlib."
>>>
>>> I disagree, people should be free to use whatever system they want.
>>
>> The above argument, in general, is invalid.
> 
> [citation needed]

Indeed.

In which moral system would your claim:

  "People should be free to use whatever system they want.",

in general, be valid?

It just puzzles me that someone can claim this.
It is like saying that:

  "People should be free do do whatever they want."

Regardless of what particular individual believes|is_able_to_explain is right or wrong, I cannot find a single system where your claim would be valid.

Re: AW: [Caml-list] GODI is shutting down

[Thomas Gazagnaire]

> On Wed, Jul 24, 2013 at 12:25 PM, Thomas Gazagnaire <thomas@ocamlpro.com> wrote:
>> However, the main problem (the one I wanted to point out) still remains: every time you want to fix a constraint you have to change your _oasis, modify the package tarball, and make a new release -- which means you can never fix constraints on already released packages. But I guess in this case, it's fine if the _oasis and OPAM files are not in sync.
> 
> The way I see it is that a package that contains incorrect dependency
> information, even if it's just a version thing, has a bug and should
> be fixed.  I agree this is annoying to package developers, but fixing
> bugs is always annoying ;-)

Unfortunately, in practice this is a nightmare as dependency constraints are mostly open: when you create a new (incompatible release), you usually invalidate existing package constraint boundaries of already released packages. For instance, let's have A, depending on B >= 1.0. Later you release B version "1.1" which breaks A. Then, you need to upgrade the dependency constraints of A to 1.0 <= B <= 1.1, ie. release a new version of A. This can in turn force you to release new packages and so on.

Cabal is dealing with that by automatically restricting the constraint boundaries[1] and I'm not very happy with that either.

--
Thomas

Re: AW: [Caml-list] GODI is shutting down

[Thomas Gazagnaire]

>>>> 6. "No, all package managers should unite in this point, and only accept
>>>> packages with oasis support. (Btw, that's homework for me.) Just do it the
>>>> same way as we did when requiring findlib."
>>>> 
>>>> I disagree, people should be free to use whatever system they want.
>>> 
>>> The above argument, in general, is invalid.
>> 
>> [citation needed]
> 
> Indeed.
> 
> In which moral system would your claim:
> 
>  "People should be free to use whatever system they want.",
> 
> in general, be valid?
> 
> It just puzzles me that someone can claim this.
> It is like saying that:
> 
>  "People should be free do do whatever they want."
> 
> Regardless of what particular individual believes|is_able_to_explain is right or wrong, I cannot find a single system where your claim would be valid.

Right, seems that people are quite keen to argue today. What I meant is: given the current state of affairs in the build system world and the very different needs of the very different set of existing users, I am *very* reluctant to force anybody to use any specific build system.

Hope this help,
Thomas

Re: [Caml-list] GODI is shutting down

[Gabriel Scherer]

> I think that's fair, but the devil is in the details. The core of OASIS
> is sound, but is hampered by its need to build on top of ocamlfind
> and ocamlbuild, and the ensuing impedance mismatches that follow (e.g.
> manually editing _tags files, and trying to debug which tool a failure
> happens in is an advanced operation). OPAM packages represent an output
> collection of (several optionally built) ocamlfind packages, and
> OASIS sits awkwardly in between these two right now, with ocamlbuild
> crashing the party and getting drunk in the kitchen.

While we're at it (and hoping a diversion might restores some
constructiveness in this discussion), I have also been thinking about
this aspect of OASIS that I don't really like. I use ocamlbuild
-use-ocamlfind, but I'd rather build my _tags and myocamlbuild.ml
files completely separately from OASIS, and avoid that intermixing of
several tools. OASIS could decide which files to build (by calling
whichever build system I use with those as targets) and include in the
user installation, but I'd rather have full control of the build
system independently.

In fact, that's more or less what there is in Batteries; we use
ocamlbuild, with a Makefile on top of it (to easily integrate some
hacks such as inline tests using
https://github.com/vincent-hugot/iTeML ), and use the XCustomBuild
option of OASIS to tell it to use the makefile. This avoids having
autogenerated stuff in the ocamlbuild setup, but currently OASIS has
no say on which files to build (it gets what "make all" produces).

In theory, integrating OASIS and the build system would allow OASIS to
diffuse best practices at the build system level as well. In practice
as a package developer I'm not fond on this combination, and I would
rather let a QA pass do the kind of checks to avoid what a tight OASIS
+ build system integration would have made impossible in the first
place (missing .cmt or what not).

PS: ocamlbuild has been getting bugfixes and improvements relatively
fast these last few months, in particular thanks to the completely
benevolous efforts of Wojchiech Meyer. I'm confident it will continue
to improve on the known-problematic fronts (terse documentation,
unfriendly failure behaviours, disappointing parallelization, etc) in
the future, which may help for some of your "fast Mirage" goals.

On Wed, Jul 24, 2013 at 6:39 PM, Anil Madhavapeddy <anil@recoil.org> wrote:
> On 24 Jul 2013, at 08:58, Markus Mottl <markus.mottl@gmail.com> wrote:
>
>> My guess is that you are not totally happy with Oasis at this point so
>> this may be a topic to address.  Being an Oasis user, I would say that
>> the Oasis tools may not be as mature yet as one would hope, but the
>> Cabal-like package specification syntax seems decent enough to be
>> supported by OPAM.  It really shouldn't be that hard to automate the
>> process of updating OPAM metadata (including version constraints)
>> based on changes in Oasis package specifications.
>
> I think that's fair, but the devil is in the details. The core of OASIS
> is sound, but is hampered by its need to build on top of ocamlfind
> and ocamlbuild, and the ensuing impedance mismatches that follow (e.g.
> manually editing _tags files, and trying to debug which tool a failure
> happens in is an advanced operation). OPAM packages represent an output
> collection of (several optionally built) ocamlfind packages, and
> OASIS sits awkwardly in between these two right now, with ocamlbuild
> crashing the party and getting drunk in the kitchen.
>
> OASIS is structured very well to help get around this situation though.
> It has a library that takes care of parsing the _oasis file, and then a
> plugin generator that currently uses ocamlbuild to drive this specification
> into concrete outputs.  Meanwhile, the OCaml compiler has been exposing
> more of its innards via compiler-libs. Leo and I are taking advantage of
> this by trying to folding all of these layers into a new "packaging-friendly"
> OCaml driver that statically links:
>
> - all the camlp4 libraries required for a project, and directly calls it
>   instead of going via -pp (avoiding a fork/marshal).
> - links to OPAM/findlib to find dependencies directly, avoiding a shell
>   out to ocamlfind.
> - can publish a static schedule of its activities to help Makefiles
>   figure out what actually got built (similar to the recent ocamlbuild
>   request).
>
> The idea is to have a "ocamlmkcompiler" that builds a binary for-purpose,
> as "ocamlmktop" does today for toplevels.  It shouldn't even need any
> patches to the official compiler, thanks to compiler-libs.  More to the
> point, it should make it far easier to build a camlp4-aware js_of_ocaml
> toplevel, to fit into the experiments that OCamlPro is doing with web
> IDE interfaces.
>
> Anyway, these are still experiments and quite likely to run into unexpected
> issues as we build them, and we'll have to consider all the alternatives
> before folding them all into OPAM.
>
> Right now, OPAM dictates the *minimal* possible set of requirements for
> a package since it just uses shell commands.  It should be possible to
> specialise this for common cases such as a OASIS library, but it does
> require some careful thought first.  Luckily, we can use OPAM itself to
> conduct such experiments [1].
>
> All such discussions and experiments are very welcome on the mailing
> list (http://lists.ocaml.org/listinfo/opam-devel) to help establish
> everyone's use cases.  Daniel Buenzli's input has been particularly
> interesting to date, as his libraries adopt a very minimal shell-script
> driven approach that doesn't use OASIS, but still distributes META files.
> That's just one example of someone that doesn't use OASIS directly.
> The Core OASIS files are also a bit special due to their configuration
> requirements.  Xen/Mirage is also quite demanding in terms of the
> compilation environment.
>
> Here's my goal: I want to have near instant-rebuilds of Mirage packages.
> I know it's possible to pull this off with the existing compiler tools,
> since the monolithic Mirage tree (with lovingly crafted ocamlbuild rules
> and judicious use of native code compiler binaries) could rebuild the
> entire webserver in ~10s in parallel.  Nowadays, with OPAM, ocamlbuild,
> ocamlfind and camlp4, it takes around 2-3 minutes, which just isn't
> good enough.
>
> -anil
>
> [1] http://opam.ocamlpro.com/pkg/debug-camlp4-log.base.html: I hacked
> up a debug camlp4 to trace all the uses of camlp4 in the OPAM repository,
> during the wg-camlp4 discussion on the topic of where to go next with it.
> This just required doing a mass OPAM install, drinking some coffee, and
> reading the resulting trace logs.
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: AW: [Caml-list] GODI is shutting down

[Thomas Gazagnaire]

missing link

> Cabal is dealing with that by automatically restricting the constraint boundaries[1] and I'm not very happy with that either.

[1] http://www.reddit.com/r/haskell/comments/f3lh5/haskells_own_dll_hell/c1d1oec

Re: AW: [Caml-list] GODI is shutting down

[Török Edwin]

On 07/24/2013 07:58 PM, Thomas Gazagnaire wrote:
>> On Wed, Jul 24, 2013 at 12:25 PM, Thomas Gazagnaire
>> <thomas@ocamlpro.com> wrote:
>>> However, the main problem (the one I wanted to point out) still
>>> remains: every time you want to fix a constraint you have to
>>> change your _oasis, modify the package tarball, and make a new
>>> release -- which means you can never fix constraints on already
>>> released packages. But I guess in this case, it's fine if the
>>> _oasis and OPAM files are not in sync.
>> 
>> The way I see it is that a package that contains incorrect
>> dependency information, even if it's just a version thing, has a
>> bug and should be fixed.  I agree this is annoying to package
>> developers, but fixing bugs is always annoying ;-)
> 
> Unfortunately, in practice this is a nightmare as dependency
> constraints are mostly open: when you create a new (incompatible
> release), you usually invalidate existing package constraint
> boundaries of already released packages. For instance, let's have A,
> depending on B >= 1.0. Later you release B version "1.1" which breaks
> A.

That is a bug in package B, and should be addressed before making v1.1 available in the main repository:
 1) either update version constraints of A as you suggest directly in opam
 2) fix package B to be backwards compatible (or at least report a bug upstream about it)
 3) do as in #1, i.e. use opam file as authoritative source, but report a bug against package A with a patch to _oasis, and a request to regenerate the files. The bugreport could even be automated (I assume package maintainers are automatically notified of bugs in their packages as well?)

There is also the possibility to automatically run 'oasis setup' for usual packages (that don't need special OASIS plugins and so on), and include the generated files as patches ... automatically.

--Edwin 

Re: [Caml-list] GODI is shutting down

[Daniel Bünzli]

Le mercredi, 24 juillet 2013 à 18:05, Gabriel Scherer a écrit :
> While we're at it (and hoping a diversion might restores some
> constructiveness in this discussion), I have also been thinking about
> this aspect of OASIS that I don't really like. I use ocamlbuild
> -use-ocamlfind, but I'd rather build my _tags and myocamlbuild.ml
> files completely separately from OASIS, and avoid that intermixing of
> several tools.  

Yes. I have always done that during development since oasis was getting in my way. I don't want that indirection. I know how to use my build system, I don't want it to be generated by someone else.

But then I would use oasis to make releases which meant expressing these build rules in oasis which is not always obvious and/or possible without still needing to have to patch your _tags file (e.g. syntax extensions) and when things were not working you were left with that 350 loc myocamlbuild.ml file and an obnoxious _tags file that you didn't write.

At a certain point I realized that oasis was actually making very few things for me: basically generating the META and being able to express conditional dependencies. When I understood a little bit more about opam, its build: section and its .install files and read about META files I realized that cost/benefit of using oasis was too high.  

Personally I'm moving away from oasis for new packages and will gradually phase out its usage during maintenance release of my old packages. The path I'm taking from now on until something better emerges is described here [1].

Best,

Daniel

[1] http://lists.ocaml.org/pipermail/opam-devel/2013-July/000161.html

Re: [Caml-list] GODI is shutting down

[Markus Mottl]

On Wed, Jul 24, 2013 at 12:39 PM, Anil Madhavapeddy <anil@recoil.org> wrote:
> I think that's fair, but the devil is in the details. The core of OASIS
> is sound, but is hampered by its need to build on top of ocamlfind
> and ocamlbuild, and the ensuing impedance mismatches that follow (e.g.
> manually editing _tags files, and trying to debug which tool a failure
> happens in is an advanced operation). OPAM packages represent an output
> collection of (several optionally built) ocamlfind packages, and
> OASIS sits awkwardly in between these two right now, with ocamlbuild
> crashing the party and getting drunk in the kitchen.

That is indeed an eloquent way of putting it :)

Oasis as a means to standardize package specifications seems important
to me, but I wish the scope of the associated tools had been less
ambitious.  The somewhat unsatisfying build system situation certainly
didn't help.

My hope is that OPAM will eventually act as the central hub that ties
all tools together and make them more seamless by enforcing (or at
least strongly supporting) some reasonable packaging and build system
standards.

Regards,
Markus

--
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com

Re: [Caml-list] GODI is shutting down

[Markus Mottl]

On Wed, Jul 24, 2013 at 1:56 PM, Daniel Bünzli
<daniel.buenzli@erratique.ch> wrote:
> But then I would use oasis to make releases which meant expressing these build rules in oasis which is not always obvious and/or possible without still needing to have to patch your _tags file (e.g. syntax extensions) and when things were not working you were left with that 350 loc myocamlbuild.ml file and an obnoxious _tags file that you didn't write.

I agree that the build process of somewhat more advanced projects is
not easy to specify with Oasis.  But for simple projects (pure OCaml,
maybe just some C-files, no preprocessing) it is decent enough.
Though I have adopted Oasis for all my open source projects, I did
sometimes despair a little when things got complicated, e.g. version
discovery of 3rd party libraries, preprocessing, etc.

> Personally I'm moving away from oasis for new packages and will gradually phase out its usage during maintenance release of my old packages. The path I'm taking from now on until something better emerges is described here [1].

I'm not going that far yet, I'll just wait and see whether Oasis will
improve to the point where the hard stuff gets easy enough, or, who
knows, until something even better comes along.  But I really do like
the idea of having a standardized packaging file within the package.
Oasis may have started out in the right place with that, but seems to
have wandered into less promising territory.

Regards,
Markus

--
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com

Re: AW: [Caml-list] GODI is shutting down

[Markus Mottl]

On Wed, Jul 24, 2013 at 1:33 PM, Török Edwin <edwin+ml-ocaml@etorok.net> wrote:
> That is a bug in package B, and should be addressed before making v1.1 available in the main repository:
>  1) either update version constraints of A as you suggest directly in opam
>  2) fix package B to be backwards compatible (or at least report a bug upstream about it)
>  3) do as in #1, i.e. use opam file as authoritative source, but report a bug against package A with a patch to _oasis, and a request to regenerate the files. The bugreport could even be automated (I assume package maintainers are automatically notified of bugs in their packages as well?)
>
> There is also the possibility to automatically run 'oasis setup' for usual packages (that don't need special OASIS plugins and so on), and include the generated files as patches ... automatically.

I think we will have to accept that on some level specified version
constraints are not authoritative.  Just because I specify in my
package that library A is compatible with whatever version of library
B doesn't make it so.  Incompatibilities may not show up during build
time, may elude all test suites, and still make your program bomb.
Full automation is hence impossible, though at least the build part
and test suite part could in principle be covered automatically.

This means that a good packaging system will have to be able to
override version constraints that are specified by a particular
package version to exclude individual versions of dependencies or
ranges thereof.  Of course, such overrides should eventually be
communicated back to the package maintainer.  This communication can
be automated: just send an email to the specified package maintainer
each time constraint overrides are added or changed, no matter whether
automatically or manually.

Regards,
Markus

--
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com

Re: [Caml-list] GODI is shutting down

[Daniel Bünzli]

Le mercredi, 24 juillet 2013 à 19:23, Markus Mottl a écrit :
> I'm not going that far yet, I'll just wait and see whether Oasis will
> improve to the point where the hard stuff gets easy enough, or, who
> knows, until something even better comes along. But I really do like
> the idea of having a standardized packaging file within the package.

Agreed, but for me that file was renamed from `_oasis` to `opam`. It has everything, the metadata, the (optional) dependencies and the build instructions.  

Besides starting with opam 1.1 if you put that opam file in the root directory of your repository, pinning a stable package to its git repo will use that `opam` file instead of the `opam` file that is provided by the stable package which may have become outdated w.r.t. to HEAD -- e.g. new dependencies were added to the project, the build procedure changed etc -- see [1].

Contrary to what Adrian Nader suggested a few emails ago pinning is absolutely not a bad idea in general. I agree with him that using unreleased software is by definition a bad idea and I'm a strong advocate of semantic versioning and formal release processes. However the pin mechanism --- which can pin to a specific *commit* --- combined with the root `opam` file in your repo, allows to: ask someone to test a fix, quickly propagate a security fix, issue a release candidate, evaluate the damage an incompatible release would entail, etc. all this without having to disturb the whole stable eco-system and with minimal energy dissipation.  

Best,

Daniel

[1] https://github.com/OCamlPro/opam/issues/671#issuecomment-21053933

Re: AW: [Caml-list] GODI is shutting down

[Siraaj Khandkar]

On Jul 24, 2013, at 10:44, Thomas Gazagnaire <thomas@ocamlpro.com> wrote:
> We have some experimental features to scan the filesystem before and after the installation, and update the corresponding `<name>.install` file with the difference but it's quite slow (because of the filesystem scanning) and this does not work when the number of jobs is greater than 1.

Would it work if you isolated the environment for each job, shadowed root dir to point to a unique temporary directory, installed there, recorded what it installed and only then copied to the real root directory?

Re: [Caml-list] GODI is shutting down

[Virgile Prevosto]

Hello,

Le mer. 24 juil. 2013 16:44:43 CEST,
Thomas Gazagnaire <thomas@ocamlpro.com> a écrit :

> > In the past days I made the experience that OPAM advocates never
> > answer to any of my objections. When I said "GODI has feature xy,
> > why doesn't OPAM has this too?" the only response is something like
> > "OPAM has this great Github-based workflow" (the most harmless
> > variant). Discussion impossible.
> 
> Let's try to be constructive then. I've tried to look for all your
> recent objections, let me known if I have missed some.

Well, I feel like there is still something that is not answered.
Agreed, this is a technical question, but it is probably the bulk of
the issue. Namely, this is the beginning of Gerd's email:

> project from the beginning. Basically, three organizations joined to  
> create a new software. The discussion about the design wasn't public.
> I was excluded from the beginning - and don't understand why, since
> I made an offer for cooperation when I first heard from Ocamlpro's
> plans (not about OPAM in particular, but other stuff, but I suggested
> to cooperate in package manager questions). I never got a response to

Presented as above, it can be summarized as "opam developers
arbitrarily refused a cooperation offer". As a potential package
maintainer, hearing this has an impact on my motivation to devote too
much time to it. I would thus be quite interested to see the other side
of this coin.

Best regards,
-- 
E tutto per oggi, a la prossima volta.
Virgile

Re: [Caml-list] GODI is shutting down

[Amir Chaudhry]

Dear all,

I don't think this thread is particularly productive and I don't see it getting any better.

The initial message said GODI is being shut down, unless someone is willing to take it over.  Since then we seem to have entered an OPAM vs GODI debate, which I find odd because unless someone has agreed to take over GODI (or Gerd changes his mind), the entire discussion is moot.  If it's about features the OCaml community feels it needs, there are also other channels where those can be discussed. 

Overall, it feels as though this thread has become more about venting frustrations and I'd question whether this list is an appropriate place for that (please realise that there are over 1300 subscribers).

Best wishes,
Amir

On 24 Jul 2013, at 23:06, Virgile Prevosto <virgile.prevosto@m4x.org> wrote:

> Hello,
> 
> Le mer. 24 juil. 2013 16:44:43 CEST,
> Thomas Gazagnaire <thomas@ocamlpro.com> a écrit :
> 
>>> In the past days I made the experience that OPAM advocates never
>>> answer to any of my objections. When I said "GODI has feature xy,
>>> why doesn't OPAM has this too?" the only response is something like
>>> "OPAM has this great Github-based workflow" (the most harmless
>>> variant). Discussion impossible.
>> 
>> Let's try to be constructive then. I've tried to look for all your
>> recent objections, let me known if I have missed some.
> 
> Well, I feel like there is still something that is not answered.
> Agreed, this is a technical question, but it is probably the bulk of
> the issue. Namely, this is the beginning of Gerd's email:
> 
>> project from the beginning. Basically, three organizations joined to  
>> create a new software. The discussion about the design wasn't public.
>> I was excluded from the beginning - and don't understand why, since
>> I made an offer for cooperation when I first heard from Ocamlpro's
>> plans (not about OPAM in particular, but other stuff, but I suggested
>> to cooperate in package manager questions). I never got a response to
> 
> Presented as above, it can be summarized as "opam developers
> arbitrarily refused a cooperation offer". As a potential package
> maintainer, hearing this has an impact on my motivation to devote too
> much time to it. I would thus be quite interested to see the other side
> of this coin.
> 
> Best regards,
> -- 
> E tutto per oggi, a la prossima volta.
> Virgile
> 
> 
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] GODI is shutting down

[Anil Madhavapeddy]

On 24 Jul 2013, at 15:06, Virgile Prevosto <virgile.prevosto@m4x.org> wrote:
> 
>> project from the beginning. Basically, three organizations joined to  
>> create a new software. The discussion about the design wasn't public.
>> I was excluded from the beginning - and don't understand why, since
>> I made an offer for cooperation when I first heard from Ocamlpro's
>> plans (not about OPAM in particular, but other stuff, but I suggested
>> to cooperate in package manager questions). I never got a response to
> 
> Presented as above, it can be summarized as "opam developers
> arbitrarily refused a cooperation offer". As a potential package
> maintainer, hearing this has an impact on my motivation to devote too
> much time to it. I would thus be quite interested to see the other side
> of this coin.

Luckily, the OPAM bug tracking speaks for itself.  As the first 'user' of
OPAM, I found it incredibly easy to work with the team.  Just inspect the
issues in reverse order and read the discussions if you want to re-live
the early, exciting days of OPAM (spoiler: they weren't very exciting).
https://github.com/OCamlPro/opam/issues?direction=asc&page=1&sort=created&state=closed

The "discussion about the design wasn't public" verges on being a farcical
statement. The design document was detailed from day 1, and Thomas kept it
updated after every major change.  Don't believe me?  Run:

$ cd opam.git
$ git log --follow doc/dev-manual/dev-manual.tex

The actual discussions of the design were very user-driven, as seen in
the issues I linked earlier, and the Mirage list which adopted OPAM early:
http://www.openmirage.org/blog/breaking-up-is-easy-with-opam 
(back then, we needed some pretty nasty compiler hacks to bootstrap
various low-level libraries, and OPAM switches did the trick nicely,
along with the remote pinning support).

Gerd persists in dismissing my "Git-workflow" point as "the most harmless
variant". What he doesn't seem to realise is that version control workflow
dominates how many industrial users interact with and fix bugs in complex
applications based on OCaml. When I coordinated the Xen repositories, I had
over 300 active repos and branches to represent all the codeflow, and today
there are around 80 OCaml repositories and libraries in Xen Cloud alone.
Jane Street publish around 40 repositories, and Mirage has around 30.

The only reason I was interested in OPAM in the beginning was this feature,
with the rest just being icing on the cake.  On the other hand, when this 
was pointed out to Gerd in 2008, his reaction was to remove Core from GODI
for QA reasons.
http://caml.inria.fr/pub/ml-archives/caml-list/2008/01/f2a634e3ce1d93a047a55316acd08b77.en.html

That's really not a helpful response. I still remember the ICFP OCaml
tutorial that Yaron and I did in 2011 with some embarrassment. It took
almost 2 hours for participants to get OCaml and libraries installed and
working.  In 2012, when we re-ran the tutorial with a very alpha version
of OPAM, it took around 30 minutes (mainly waiting for compiler builds to
finish). This year, I expect it to be a non-issue.  The year after, users
should have binaries packages to make it near-instant.

That's progress of the good kind.  Time for a group hug!

-anil

Re: [Caml-list] GODI is shutting down

[Francois Berenger]

On 07/25/2013 02:05 AM, Gabriel Scherer wrote:
>> I think that's fair, but the devil is in the details. The core of OASIS
>> is sound, but is hampered by its need to build on top of ocamlfind
>> and ocamlbuild, and the ensuing impedance mismatches that follow (e.g.
>> manually editing _tags files, and trying to debug which tool a failure
>> happens in is an advanced operation). OPAM packages represent an output
>> collection of (several optionally built) ocamlfind packages, and
>> OASIS sits awkwardly in between these two right now, with ocamlbuild
>> crashing the party and getting drunk in the kitchen.
>
> While we're at it (and hoping a diversion might restores some
> constructiveness in this discussion), I have also been thinking about
> this aspect of OASIS that I don't really like. I use ocamlbuild
> -use-ocamlfind, but I'd rather build my _tags and myocamlbuild.ml
> files completely separately from OASIS

Personnally, I'd rather have a single human-readable file containing all 
the build information.

With the current setup in oasis used by a developper, you get:
a _tags file, a myocamlbuild.ml and you need a Makefile
on top of that (with some parts automatically generated) to pass more 
options to ocamlbuild.
Amazing!
But at least I see a clear culprit (ocamlbuild).

Regards,
F.

Re: [Caml-list] GODI is shutting down

[Adrien Nader]

On Wed, Jul 24, 2013, Anil Madhavapeddy wrote:
> The only reason I was interested in OPAM in the beginning was this feature,
> with the rest just being icing on the cake.  On the other hand, when this 
> was pointed out to Gerd in 2008, his reaction was to remove Core from GODI
> for QA reasons.
> http://caml.inria.fr/pub/ml-archives/caml-list/2008/01/f2a634e3ce1d93a047a55316acd08b77.en.html

Unless I'm mistaken, Core was still in the godi packages not so long
ago. This message is from January 2008. That's a fairly long delay for a
removal.

-- 
Adrien Nader

Re: [Caml-list] GODI is shutting down

[Adrien Nader]

On Wed, Jul 24, 2013, Daniel Bünzli wrote:
> Le mercredi, 24 juillet 2013 à 19:23, Markus Mottl a écrit :
> > I'm not going that far yet, I'll just wait and see whether Oasis will
> > improve to the point where the hard stuff gets easy enough, or, who
> > knows, until something even better comes along. But I really do like
> > the idea of having a standardized packaging file within the package.
> 
> Agreed, but for me that file was renamed from `_oasis` to `opam`. It has everything, the metadata, the (optional) dependencies and the build instructions.  

But that doesn't match.
OPAM is a source-based package manager. It needs the compiler to work.
Maybe that it will support binary packages but today it isn't there and
it will take some time for this feature to appear of course and then the
packages will have to be made, and there's the question of who or what
will be the builder and whether it can be trusted or not.

OPAM will not replace a distribution's package manager. It's good for
people who develop or are somehow involved in the language but for
everyone else, what they want and need is packages for their
distribution. That was the goal for oasis, that makes no sense as a goal
for opam.

> Contrary to what Adrian Nader suggested a few emails ago pinning is absolutely not a bad idea in general. I agree with him that using unreleased software is by definition a bad idea and I'm a strong advocate of semantic versioning and formal release processes. However the pin mechanism --- which can pin to a specific *commit* --- combined with the root `opam` file in your repo, allows to: ask someone to test a fix, quickly propagate a security fix, issue a release candidate, evaluate the damage an incompatible release would entail, etc. all this without having to disturb the whole stable eco-system and with minimal energy dissipation.  

I probably expressed myself poorly. Pinning is good when you want to do
a specific task at a specific time. It's not good when you start pinning
several things at once and keep them that way for a long time. In other
words, what I meant was that pinning of several repos was not a
substitute for releases.

-- 
Adrien Nader

Re: [Caml-list] GODI is shutting down

[Daniel Bünzli]

> OPAM is a source-based package manager. It needs the compiler to work.
> Maybe that it will support binary packages but today it isn't there and
> it will take some time for this feature to appear of course and then the
> packages will have to be made, and there's the question of who or what
> will be the builder and whether it can be trusted or not.

oasis doesn't need the compiler to work ? Did you ever use an oasis based package ? 
 
> OPAM will not replace a distribution's package manager. 

Who said it would ? And oasis doesn't either.

> It's good for people who develop or are somehow involved in the language but for
> everyone else, what they want and need is packages for their
> distribution. That was the goal for oasis, that makes no sense as a goal
> for opam.

No. From the oasis page: 

"OASIS is a tool to integrate a configure, build and install system in your OCaml project. It helps to create standard entry points in your build system and allows external tools to analyse your project easily."

It was a meta build thing. If you do a meta thing do it lightly and obviously or fail. It failed, even in being meta, it still supports only ocamlbuild and in an unsatisfying way. 

An opam .install file allows you to trivially express more (e.g. man pages to install, things to put in share) than what oasis is able after all these years. If your build system takes care to generate this file (or provide it statically if your project is not complex) it can be used by both opam and distribution packagers to understand what needs to be installed. 

Best,

Daniel

[Caml-list] Re: GODI is shutting down

[Sylvain Le Gall]

On 24-07-2013, Daniel Bünzli <daniel.buenzli@erratique.ch> wrote:
> Le mercredi, 24 juillet 2013 à 09:42, Jun Furuse a écrit :
>> Have you discussed your idea of OPAM with Gerd or GODI community before starting the project? Was there no way of working together, to improve GODI, or invite Gerd to OPAM development?
>
>
> I find it strange that for quite sometime we had other individuals
> working on a similar concurrent project (oasis-db) and nobody every
> screamed at them. That project never delivered and now that we have
> people actually delivering we should scream at them ?
>
> I tried godi a very long time ago and was not happy about the
> centralized workflow, maybe that changed I don't know. I then used odb
> for sometime because it was working for the simple things I did and it
> was easy to hack with or provide your own package sources but I still
> saw it as a quick hack --- albeit a very useful one for the time it
> served me, thanks to Edgar Friendly.
>

Please don't mix everything:
1. oasis-db wasn't meant to replace GODI (maybe the Hump if you really
want to compare)
2. as a matter of fact you were using odb and seems to be happy -- never
forget that part of odb rely on oasis-db (which was a short but fruitful
collaboration with thelema)

There was plan to indeed have a oasis-db do something similar to GODI,
but I was not convinced that it should be done (and actually spend
almost no time on it).

Please, this thread is about GODI/opam, I really don't want to be
involved. 

Cheers,
Sylvain Le Gall
--
Website:     http://sylvain.le-gall.net/
OCaml forge: http://forge.ocamlcore.org
OCaml blogs: http://planet.ocaml.org

[Caml-list] Re: AW: GODI is shutting down

[Sylvain Le Gall]

On 24-07-2013, Thomas Gazagnaire <thomas@ocamlpro.com> wrote:
>>>>> 6. "No, all package managers should unite in this point, and only accept
>>>>> packages with oasis support. (Btw, that's homework for me.) Just do it the
>>>>> same way as we did when requiring findlib."
>>>>> 
>>>>> I disagree, people should be free to use whatever system they want.
>>>> 
>>>> The above argument, in general, is invalid.
>>> 
>>> [citation needed]
>> 
>> Indeed.
>> 
>> In which moral system would your claim:
>> 
>>  "People should be free to use whatever system they want.",
>> 
>> in general, be valid?
>> 
>> It just puzzles me that someone can claim this.
>> It is like saying that:
>> 
>>  "People should be free do do whatever they want."
>> 
>> Regardless of what particular individual believes|is_able_to_explain is right or wrong, I cannot find a single system where your claim would be valid.
>
> Right, seems that people are quite keen to argue today. What I meant
> is: given the current state of affairs in the build system world and
> the very different needs of the very different set of existing users,
> I am *very* reluctant to force anybody to use any specific build
> system.
>

OASIS is not a build system! It is a kind of makemaker (or
autoconf/automake).

The default target is to generate a build system for ocamlbuild. There
is also the options to use simple command to build and to e.g. use a
Makefile base system.

I am also reluctant to integrate other build system as target (e.g
OMake).

The good point about OASIS is that it gives you a upstream dev
definition of dependencies and constraints. This is a nice first step to
generate whatever package. I had some successes generating GODI, OPAM and
Debian packages.

OASIS has been designed so that it can be useful to upstream developper
and to packager. Upstream developper can use it to build their software
and packager can extract the very same file (_oasis) used by upstream to
generate package. Generally speaking this was a good way to solve the
obsolescene of META file (because upstream devs don't need a META it is
only used by end user, so it tends to be not up to date).

The good practice should be that if OPAM QA system find a bad
dependencies, it should tell the upstream to fix its dependencies
definition in _oasis and do a release so that all other packagers will
benefit from the bug found.

Cheers,
Sylvain Le Gall
--
Website:     http://sylvain.le-gall.net/
OCaml forge: http://forge.ocamlcore.org
OCaml blogs: http://planet.ocaml.org

[Caml-list] Re: GODI is shutting down

[Sylvain Le Gall]

On 25-07-2013, Francois Berenger <berenger@riken.jp> wrote:
> On 07/25/2013 02:05 AM, Gabriel Scherer wrote:
>>> I think that's fair, but the devil is in the details. The core of OASIS
>>> is sound, but is hampered by its need to build on top of ocamlfind
>>> and ocamlbuild, and the ensuing impedance mismatches that follow (e.g.
>>> manually editing _tags files, and trying to debug which tool a failure
>>> happens in is an advanced operation). OPAM packages represent an output
>>> collection of (several optionally built) ocamlfind packages, and
>>> OASIS sits awkwardly in between these two right now, with ocamlbuild
>>> crashing the party and getting drunk in the kitchen.
>>
>> While we're at it (and hoping a diversion might restores some
>> constructiveness in this discussion), I have also been thinking about
>> this aspect of OASIS that I don't really like. I use ocamlbuild
>> -use-ocamlfind, but I'd rather build my _tags and myocamlbuild.ml
>> files completely separately from OASIS
>
> Personnally, I'd rather have a single human-readable file containing all 
> the build information.
>
> With the current setup in oasis used by a developper, you get:
> a _tags file, a myocamlbuild.ml and you need a Makefile
> on top of that (with some parts automatically generated) to pass more 
> options to ocamlbuild.
> Amazing!
> But at least I see a clear culprit (ocamlbuild).
>

It depends a lot. If you use
$> oasis setup -setup-update dynamic

You generate a very small set of files (no _tags and no myocamlbuild if
they are not here). I use that for certain project.

I think your problem is that OASIS try to generate standalone
installation (i.e. a package that doesn't depend on OASIS to build),
which means I have to pre-generate files for ocamlbuild, just as
automake...

Now, if the community as a whole say that we don't need a standalone
generation and can rely on OASIS to build, it would make my life 10x
easier (getting rid of super-complex 2-stage setup, ocaml-data-notation,
ocamlmod, ocamlify...)

Cheers,
Sylvain Le Gall
-- 
Website:     http://sylvain.le-gall.net/
OCaml forge: http://forge.ocamlcore.org
OCaml blogs: http://planet.ocaml.org

[Caml-list] Re: AW: GODI is shutting down

[Sylvain Le Gall]

On 24-07-2013, Thomas Gazagnaire <thomas@ocamlpro.com> wrote:
>> Oasis may are may not become the standard OCaml package specification
>> tool, but it is widely used so I think it's fair if it is better
>> supported by OPAM than alternatives.
>
> Right, this is indeed a feature requested by many people, than I can hear. This is tracked and discussed in [1] and I'm fine to try to implement the proposed solution in 1.1 (I've just updated the issue's release target).
>
> [1] https://github.com/OCamlPro/opam/issues/156
>
>> It really shouldn't be that hard to automate the
>> process of updating OPAM metadata (including version constraints)
>> based on changes in Oasis package specifications.
>
> A substantial amount of work has already been done by Christophe Troestler to automate the conversion of _oasis  into OPAM files:
>
>     $ opam install oasis2opam
>
> I'm sure this can be completed to automate the pull-request machinery as well.
>
> However, the main problem (the one I wanted to point out) still
> remains: every time you want to fix a constraint you have to change
> your _oasis, modify the package tarball, and make a new release --
> which means you can never fix constraints on already released
> packages. But I guess in this case, it's fine if the _oasis and OPAM
> files are not in sync.
>

We don't share the same POV here. If a constraints is bad it should be
fixe in the upstream version. A quick fix could be to patch it in OPAM,
but if a constraint is bad with OPAM, Debian packagers will have to do
the same kind of fix. So we should make it upsteam, not keep the fix in
OPAM.

Cheers,
Sylvain Le Gall
--
Website:     http://sylvain.le-gall.net/
OCaml forge: http://forge.ocamlcore.org
OCaml blogs: http://planet.ocaml.org

[Caml-list] Re: GODI is shutting down

[Sylvain Le Gall]

On 24-07-2013, Anil Madhavapeddy <anil@recoil.org> wrote:
> On 24 Jul 2013, at 09:36, Gabriel Scherer <gabriel.scherer@gmail.com> wrote:
>
>>> However, the main problem (the one I wanted to point out) still remains: every time you want to fix a constraint you have to change your _oasis, modify the package tarball, and make a new release -- which means you can never fix constraints on already released packages. But I guess in this case, it's fine if the _oasis and OPAM files are not in sync.
>> 
>> You could include a patch against the _oasis file in the OPAM (or
>> GODI) metadata.
>
> That's not enough.  You also need a build-time requirement to regenerate
> the OASIS autogenerated files by calling 'oasis setup', which increases
> the total build time significantly.
>
> Some projects (such as Mirage) also use a forked OASIS to add some
> specialised features in the generated output, and so need special rules
> to regenerate them.
>

Are you talking about the the output-obj ?

Your pull request is pending and I am more than willing to integrate it,
but given the copyright and license you put, I cannot... Please fix it
and stop having a branch of OASIS for that.
https://github.com/ocaml/oasis/pull/7

Cheers,
Sylvain Le Gall
-- 
Website:     http://sylvain.le-gall.net/
OCaml forge: http://forge.ocamlcore.org
OCaml blogs: http://planet.ocaml.org

[Caml-list] Re: GODI is shutting down

[Christopher Zimmermann]

On Thu, 25 Jul 2013 15:10:37 +0000 (UTC)
Sylvain Le Gall <sylvain@le-gall.net> wrote:

> Now, if the community as a whole say that we don't need a standalone
> generation and can rely on OASIS to build, it would make my life 10x
> easier (getting rid of super-complex 2-stage setup,
> ocaml-data-notation, ocamlmod, ocamlify...)

I'm doing a lot of OCaml porting work on OpenBSD.
I did not port OASIS for a very long time because of its comprehensive
dependencies. The only reason for porting OASIS at last was that I
needed to patch _oasis on a port.
Still from my perspective I vote for getting rid of the 2-stage setup
for two reasons:

1. porting ocaml-data-notation, ocamlmod, ocamlify was annoying
enough when all I wanted to port was oasis.
2. This way the porters will be forced to port oasis earlier, but this
job will also be a lot easier.


Christopher

-- 
OpenPGP public key: http://gmerlin.de/christopher.pub
fingerprint: 1917 680A 723C BF3D 2CA3  0E44 7E24 D19F 34B8 2A2A

Re: [Caml-list] Re: AW: GODI is shutting down

[Leo White]

>> However, the main problem (the one I wanted to point out) still
>> remains: every time you want to fix a constraint you have to change
>> your _oasis, modify the package tarball, and make a new release --
>> which means you can never fix constraints on already released
>> packages. But I guess in this case, it's fine if the _oasis and OPAM
>> files are not in sync.
>>
>
> We don't share the same POV here. If a constraints is bad it should be
> fixe in the upstream version. A quick fix could be to patch it in OPAM,
> but if a constraint is bad with OPAM, Debian packagers will have to do
> the same kind of fix. So we should make it upsteam, not keep the fix in
> OPAM.
>

I think that Thomas' point was that if you extract the dependecies
directly from an _oasis file within the package then you can only fix
dependencies in future versions of the package.

So, if foo-0.5.6 has a bad dependency (extracted from the _oasis file
within foo-0.5.6) making the fix upstream will only help foo-0.5.7
onwards: foo-0.5.6 will still be in the repository and its dependencies
will still be incorrect.

I am sure he agress that fixes should be passed upstream, but we also
need to be able to correct dependencies in existing packages.

Regards,

Leo

[Caml-list] Re: AW: GODI is shutting down

[Sylvain Le Gall]

On 25-07-2013, Leo White <lpw25@cam.ac.uk> wrote:
>>> However, the main problem (the one I wanted to point out) still
>>> remains: every time you want to fix a constraint you have to change
>>> your _oasis, modify the package tarball, and make a new release --
>>> which means you can never fix constraints on already released
>>> packages. But I guess in this case, it's fine if the _oasis and OPAM
>>> files are not in sync.
>>>
>>
>> We don't share the same POV here. If a constraints is bad it should be
>> fixe in the upstream version. A quick fix could be to patch it in OPAM,
>> but if a constraint is bad with OPAM, Debian packagers will have to do
>> the same kind of fix. So we should make it upsteam, not keep the fix in
>> OPAM.
>>
>
> I think that Thomas' point was that if you extract the dependecies
> directly from an _oasis file within the package then you can only fix
> dependencies in future versions of the package.
>
> So, if foo-0.5.6 has a bad dependency (extracted from the _oasis file
> within foo-0.5.6) making the fix upstream will only help foo-0.5.7
> onwards: foo-0.5.6 will still be in the repository and its dependencies
> will still be incorrect.
>
> I am sure he agress that fixes should be passed upstream, but we also
> need to be able to correct dependencies in existing packages.
>

Getting out of sync for older packages seems fine to me! The system
should allow override and that should be ok. 

At least we agree on this point!

Cheers,
Sylvain Le Gall
-- 
Website:     http://sylvain.le-gall.net/
OCaml forge: http://forge.ocamlcore.org
OCaml blogs: http://planet.ocaml.org

[Caml-list] Re: GODI is shutting down

[Sylvain Le Gall]

Hello all,

This whole discussion has drifted into various technical topics but I
would like to give my POV on the general topic of OCamlPro vs some
members of the community.

I think OPAM is a great piece of work and I wished we can have two
package manager, including GODI for which I have contributed a little.

OCamlPro has the worforce to do things that most of the individuals
cannot do: have a team of engineers working on an OCaml topic. That's
great expect that when you know your software is on the list of topics.
When you are a team of 1 and OCamlPro has decided to work on something
close to what you are doing, you know that you should stop working on it
because you cannot compete (lack of time, lack of workforce).

This kind of feeling makes me extremly uncomfortable about the future of
all the libraries I maintain. This kind of FUD is extremly
counter-productive for any OCaml dev.

To give you a quick taste of that, you can read:
http://hal.inria.fr/docs/00/66/59/62/PDF/paper_10.pdf (fr)

where you can see that ocp-build is supposed to replace OASIS
(explanation about why OASIS doesn't fit, p2 at the beginning) and that
it should replace findlib's META file (p12, Conclusion).

Some people will see a good news in this document and some people will
feel uncomfortable (like me).

I don't think making some people feel uncomfortable is great.

Given OCamlPro position, I would love to see them achieve great things
and solve long standing issue like:
https://github.com/lucasaiu/ocaml (a reentrant runtime which should be
the stepping stone for parallelism in OCaml)

I think it would be a lot more productive than to replace every OCaml
tools by something starting with ocp-...

My 0.02 cents
Sylvain

On 22-07-2013, Gerd Stolpmann <info@gerd-stolpmann.de> wrote:
> Dear subscriber,
>
> Unfortunately, it is no longer possible for me to run the GODI
> distribution. GODI will not upgrade to OCaml 4.01 once it is out, and it
> will shut down the public service in the course of September 2013. The
> website, camlcity.org, will remain up, but with reduced content. Existing
> GODI installations can be continued to be used, but upgrades or bugfixes
> will not be available when GODI is off.
>
> Although there are still a lot of GODI users, it is unavoidable to shut
> GODI down due to lack of supporters, especially package developers. I was
> more or less alone in the past months, and my time contingent will not
> allow it to do the upgrade to OCaml 4.01 alone (when it is released).
>
> Also, there was a lot of noise about a competing packaging system for
> OCaml in the past weeks: OPAM. Apparently, it got a lot of attention both
> from individuals and from organizations. As I see it, the OCaml community
> is too small to support two systems, and so in some sense GODI is
> displaced by OPAM.
>
> The sad part is that OPAM is only clearly better in one point, namely in
> interacting with the community (via Github). In times where social
> networks are worth billions this is probably the striking point. It
> doesn't matter that OPAM lacks core functions like deleting all files when
> a package is removed, and that it lacks many other features GODI has. So
> there is some loss of functionality for the community (partly difficult to
> replace, like GODI's support for Windows).
>
> If somebody wants to take over GODI, please do so. The source code is
> still available as well as the package directories. Maybe it is sufficient
> to move the repository to a public place and to redesign the package
> release process to give GODI a restart.
>
> There is also another point that was driving me mad in the past weeks,
> namely missing respect from the OPAM guys. Given the fact that OPAM is
> only a thin layer around ocamlfind (and guess who wrote it), and given the
> fact that GODI was pioneering in many fields, I was expecting nicer
> wordings, and less dumb campaigning ("we have 400 packages, and you only
> 170"). OPAM is only harvesting what I seeded many years ago.
>
> Let's hope these guys get now some kicks into their asses, and are forced
> to add all the functionality to OPAM the OCaml community deserves.
>
> Hoorn (NL), the 22nd July 2013,
>
-- 
Website:     http://sylvain.le-gall.net/
OCaml forge: http://forge.ocamlcore.org
OCaml blogs: http://planet.ocaml.org

Re: [Caml-list] Re: GODI is shutting down

[Daniel Bünzli]

> I don't think making some people feel uncomfortable is great.
Personally I don't choose my tools based on politeness, I choose them from the perspective of technical excellence, usability, licensing and maintenance.

If OCamlPro produces software that matches these criterions I see absolutely no reason of not using it even if it renders other tools obsolete and I'm actually very glad OCamlPro exists at the moment to inject energy into building a better OCaml environment. 

As far as I'm concerned they produced opam, ocp-indent and ocp-index which are tools I use every day and I'm very happy with --- I'm not convinced by ocp-build so far but I don't mind using if it eventually matches my needs.

What I mostly see in the "general topic of OCamlPro vs some members of the community" (your words) is ego problems. 

Best,

Daniel








 

Re: [Caml-list] Re: GODI is shutting down

[Fabrice Le Fessant]

[-- Attachment #1: Type: text/plain, Size: 2815 bytes --]

Hi Sylvain,

I am sorry you feel that way.

For me, it's actually the opposite: OCamlPro tries to improve the whole
OCaml development infrastructure, adding features and optimizations to the
compiler, writing new generic libraries and development tools, and taking
the burden of maintaining them on the long term, so that all of us, OCaml
developers and companies, can focus on hacking our favorite application in
OCaml, without having first to rewrite all the tools you usually find with
other languages.

Also, as a service-provider, we mostly only work on tools and libraries
that are funded by our customers, because they think that these tools are
important to their business. Our current workforce on this topic is not so
big, we could do much more to improve the environment, but for that, we
need to find more customers (hint :-) ) so that we can design the best
solutions tailored to their needs.

--Fabrice
Scientific advisor for OCamlPro

On Thu, Jul 25, 2013 at 6:11 PM, Sylvain Le Gall <sylvain@le-gall.net>wrote:

> This whole discussion has drifted into various technical topics but I
> would like to give my POV on the general topic of OCamlPro vs some
> members of the community.


> I think OPAM is a great piece of work and I wished we can have two
> package manager, including GODI for which I have contributed a little.
>
> OCamlPro has the worforce to do things that most of the individuals
> cannot do: have a team of engineers working on an OCaml topic. That's
> great expect that when you know your software is on the list of topics.
> When you are a team of 1 and OCamlPro has decided to work on something
> close to what you are doing, you know that you should stop working on it
> because you cannot compete (lack of time, lack of workforce).
>
> This kind of feeling makes me extremly uncomfortable about the future of
> all the libraries I maintain. This kind of FUD is extremly
> counter-productive for any OCaml dev.
>
> To give you a quick taste of that, you can read:
> http://hal.inria.fr/docs/00/66/59/62/PDF/paper_10.pdf (fr)
>
> where you can see that ocp-build is supposed to replace OASIS
> (explanation about why OASIS doesn't fit, p2 at the beginning) and that
> it should replace findlib's META file (p12, Conclusion).
>
> Some people will see a good news in this document and some people will
> feel uncomfortable (like me).
>
> I don't think making some people feel uncomfortable is great.
>
> Given OCamlPro position, I would love to see them achieve great things
> and solve long standing issue like:
> https://github.com/lucasaiu/ocaml (a reentrant runtime which should be
> the stepping stone for parallelism in OCaml)
>
> I think it would be a lot more productive than to replace every OCaml
> tools by something starting with ocp-...
>
> My 0.02 cents
> Sylvain
>

[-- Attachment #2: Type: text/html, Size: 3757 bytes --]

Re: [Caml-list] Re: GODI is shutting down

[Sylvain Le Gall]

2013/7/25 Daniel Bünzli <daniel.buenzli@erratique.ch>:
>> I don't think making some people feel uncomfortable is great.
> Personally I don't choose my tools based on politeness, I choose them from the perspective of technical excellence, usability, licensing and maintenance.
>
> If OCamlPro produces software that matches these criterions I see absolutely no reason of not using it even if it renders other tools obsolete and I'm actually very glad OCamlPro exists at the moment to inject energy into building a better OCaml environment.
>
> As far as I'm concerned they produced opam, ocp-indent and ocp-index which are tools I use every day and I'm very happy with --- I'm not convinced by ocp-build so far but I don't mind using if it eventually matches my needs.
>
> What I mostly see in the "general topic of OCamlPro vs some members of the community" (your words) is ego problems.

Totally agree that is an of ego problem. I hope it was clear enough in
my first mail. The topic is probably more about community management
than technical stuff.

I think OCamlPro do a great technical job. I just wish they improve
their PR to be not only "good technically" but also "have a good
leadership". The OCaml community is too small to split it even more
because of this kind of conflict.

On the long term, having good leadership will help us (re)grow the
OCaml community and that should be a goal for all of us.

Cheers
Sylvain

Re: [Caml-list] Re: GODI is shutting down

[Sylvain Le Gall]

Hi,

2013/7/25 Fabrice Le Fessant <Fabrice.Le_fessant@inria.fr>:
> Hi Sylvain,
>
> I am sorry you feel that way.
>
> For me, it's actually the opposite: OCamlPro tries to improve the whole
> OCaml development infrastructure, adding features and optimizations to the
> compiler, writing new generic libraries and development tools, and taking
> the burden of maintaining them on the long term, so that all of us, OCaml
> developers and companies, can focus on hacking our favorite application in
> OCaml, without having first to rewrite all the tools you usually find with
> other languages.
>
> Also, as a service-provider, we mostly only work on tools and libraries that
> are funded by our customers, because they think that these tools are
> important to their business. Our current workforce on this topic is not so
> big, we could do much more to improve the environment, but for that, we need
> to find more customers (hint :-) ) so that we can design the best solutions
> tailored to their needs.
>

Don't miss my point. I think OCamlPro is achieving a technical
challenge (although I wish OCamlPro will improve parallelism in OCaml
rather than something else). I just wish you make it with a good
leadership: making people talk together and collaborating. It will
probably just cost you a few emails to do it and it is not like you
are not aware of other people existing in the OCaml community.

Dev. behind closed door is a way to manage a company, but when it came
to OSS community it quickly makes member of the community feel weird.

OCamlPro should just improve its leadership and I'll be fine.

Regards
Sylvain

> --Fabrice
> Scientific advisor for OCamlPro
>
> On Thu, Jul 25, 2013 at 6:11 PM, Sylvain Le Gall <sylvain@le-gall.net>
> wrote:
>>
>> This whole discussion has drifted into various technical topics but I
>> would like to give my POV on the general topic of OCamlPro vs some
>> members of the community.
>>
>>
>> I think OPAM is a great piece of work and I wished we can have two
>> package manager, including GODI for which I have contributed a little.
>>
>> OCamlPro has the worforce to do things that most of the individuals
>> cannot do: have a team of engineers working on an OCaml topic. That's
>> great expect that when you know your software is on the list of topics.
>> When you are a team of 1 and OCamlPro has decided to work on something
>> close to what you are doing, you know that you should stop working on it
>> because you cannot compete (lack of time, lack of workforce).
>>
>> This kind of feeling makes me extremly uncomfortable about the future of
>> all the libraries I maintain. This kind of FUD is extremly
>> counter-productive for any OCaml dev.
>>
>> To give you a quick taste of that, you can read:
>> http://hal.inria.fr/docs/00/66/59/62/PDF/paper_10.pdf (fr)
>>
>> where you can see that ocp-build is supposed to replace OASIS
>> (explanation about why OASIS doesn't fit, p2 at the beginning) and that
>> it should replace findlib's META file (p12, Conclusion).
>>
>> Some people will see a good news in this document and some people will
>> feel uncomfortable (like me).
>>
>>
>> I don't think making some people feel uncomfortable is great.
>>
>> Given OCamlPro position, I would love to see them achieve great things
>> and solve long standing issue like:
>> https://github.com/lucasaiu/ocaml (a reentrant runtime which should be
>> the stepping stone for parallelism in OCaml)
>>
>> I think it would be a lot more productive than to replace every OCaml
>> tools by something starting with ocp-...
>>
>> My 0.02 cents
>> Sylvain
>
>

Re: [Caml-list] Re: GODI is shutting down

[Yotam Barnoy]

> Don't miss my point. I think OCamlPro is achieving a technical
> challenge (although I wish OCamlPro will improve parallelism in OCaml
> rather than something else). I just wish you make it with a good
> leadership: making people talk together and collaborating. It will
> probably just cost you a few emails to do it and it is not like you
> are not aware of other people existing in the OCaml community.
>

I've been 'lurking' throughout this discussion (and I'm sure many
others have as well), but I want to second this point. I use ocaml in
an academic setting, but my group is migrating away from it mostly
because haskell offers parallelism. The future of ocaml rests on it
having a parallel runtime and a parallel GC. I know these things
aren't easy to do, but nothing else will stem the migration to F# and
haskell IMO.  This is exactly the kind of problem that's very hard for
OSS devs to address. Haskell has only been able to make the huge
strides it has in terms of parallelism and performance because of the
constant work of Simon Peyton Jones and Simon Marlow with Microsoft's
generous backing. That's why I'm surprised that OcamlPro and OcamlLabs
aren't putting virtually every engineer/researcher they have on
solving this problem. I suppose the reason for this is that their
corporate backers are happy with an Async-like, process-based
execution model. However, this is short sighted IMO, as without a
parallel runtime, the language simply cannot compete, which is a
shame, because I think that WITH a parallel runtime, ocaml would beat
just about every other high level language in terms of speed.

Yotam Barnoy

Re: [Caml-list] Re: GODI is shutting down

[Adrien Nader]

Hi,

I believe this entry on the bugtracker is worth mentionning:
  "ocamlbuild should expose common interface as a library"
  http://caml.inria.fr/mantis/view.php?id=5869

More generally, ocamlbuild's development was mostly stalled but has
restarted recently. This should allow fixing many issues and
shortcomings.

-- 
Adrien Nader

Re: [Caml-list] GODI is shutting down

[Wojciech Meyer]

I want to say, at any point, ocamlbuild will get fixes and improvements
from the maintainers (mostly Gabriel Scherer and me), but we also look
quite much for the community patches. Nothing makes our life easier than
a good patch improving the tool or fixing some problem. This is a gentle
encouragement for those who would like to contribute but somewhat think
that will hit the wall of bureaucracy. We usually act quite fast if the
patch submitted is well prepared. OCamlbuild has a huge potential, and
will remain maintained. We are also thinking about larger developments,
(improving performance, tighter ocamlfind integration, exposing the
interface) but pretty much we would like to discuss it with the
community, to ensure it's that what we want.

Gabriel Scherer <gabriel.scherer@gmail.com> writes:

>>> I think that's fair, but the devil is in the details. The core of OASIS
>>> is sound, but is hampered by its need to build on top of ocamlfind
>>> and ocamlbuild, and the ensuing impedance mismatches that follow (e.g.
>>> manually editing _tags files, and trying to debug which tool a failure
>>> happens in is an advanced operation). OPAM packages represent an output
>>> collection of (several optionally built) ocamlfind packages, and
>>> OASIS sits awkwardly in between these two right now, with ocamlbuild
>>> crashing the party and getting drunk in the kitchen.
>>
>> While we're at it (and hoping a diversion might restores some
>> constructiveness in this discussion), I have also been thinking about
>> this aspect of OASIS that I don't really like. I use ocamlbuild
>> -use-ocamlfind, but I'd rather build my _tags and myocamlbuild.ml
>> files completely separately from OASIS, and avoid that intermixing of
>> several tools. OASIS could decide which files to build (by calling
>> whichever build system I use with those as targets) and include in the
>> user installation, but I'd rather have full control of the build
>> system independently.
>>
>> In fact, that's more or less what there is in Batteries; we use
>> ocamlbuild, with a Makefile on top of it (to easily integrate some
>> hacks such as inline tests using
>> https://github.com/vincent-hugot/iTeML ), and use the XCustomBuild
>> option of OASIS to tell it to use the makefile. This avoids having
>> autogenerated stuff in the ocamlbuild setup, but currently OASIS has
>> no say on which files to build (it gets what "make all" produces).
>>
>> In theory, integrating OASIS and the build system would allow OASIS to
>> diffuse best practices at the build system level as well. In practice
>> as a package developer I'm not fond on this combination, and I would
>> rather let a QA pass do the kind of checks to avoid what a tight OASIS
>> + build system integration would have made impossible in the first
>> place (missing .cmt or what not).
>>
>> PS: ocamlbuild has been getting bugfixes and improvements relatively
>> fast these last few months, in particular thanks to the completely
>> benevolous efforts of Wojchiech Meyer. I'm confident it will continue
>> to improve on the known-problematic fronts (terse documentation,
>> unfriendly failure behaviours, disappointing parallelization, etc) in
>> the future, which may help for some of your "fast Mirage" goals.
>>
>> On Wed, Jul 24, 2013 at 6:39 PM, Anil Madhavapeddy <anil@recoil.org> wrote:
>>> On 24 Jul 2013, at 08:58, Markus Mottl <markus.mottl@gmail.com> wrote:
>>>
>>>> My guess is that you are not totally happy with Oasis at this point so
>>>> this may be a topic to address.  Being an Oasis user, I would say that
>>>> the Oasis tools may not be as mature yet as one would hope, but the
>>>> Cabal-like package specification syntax seems decent enough to be
>>>> supported by OPAM.  It really shouldn't be that hard to automate the
>>>> process of updating OPAM metadata (including version constraints)
>>>> based on changes in Oasis package specifications.
>>>
>>> I think that's fair, but the devil is in the details. The core of OASIS
>>> is sound, but is hampered by its need to build on top of ocamlfind
>>> and ocamlbuild, and the ensuing impedance mismatches that follow (e.g.
>>> manually editing _tags files, and trying to debug which tool a failure
>>> happens in is an advanced operation). OPAM packages represent an output
>>> collection of (several optionally built) ocamlfind packages, and
>>> OASIS sits awkwardly in between these two right now, with ocamlbuild
>>> crashing the party and getting drunk in the kitchen.
>>>
>>> OASIS is structured very well to help get around this situation though.
>>> It has a library that takes care of parsing the _oasis file, and then a
>>> plugin generator that currently uses ocamlbuild to drive this specification
>>> into concrete outputs.  Meanwhile, the OCaml compiler has been exposing
>>> more of its innards via compiler-libs. Leo and I are taking advantage of
>>> this by trying to folding all of these layers into a new "packaging-friendly"
>>> OCaml driver that statically links:
>>>
>>> - all the camlp4 libraries required for a project, and directly calls it
>>>   instead of going via -pp (avoiding a fork/marshal).
>>> - links to OPAM/findlib to find dependencies directly, avoiding a shell
>>>   out to ocamlfind.
>>> - can publish a static schedule of its activities to help Makefiles
>>>   figure out what actually got built (similar to the recent ocamlbuild
>>>   request).
>>>
>>> The idea is to have a "ocamlmkcompiler" that builds a binary for-purpose,
>>> as "ocamlmktop" does today for toplevels.  It shouldn't even need any
>>> patches to the official compiler, thanks to compiler-libs.  More to the
>>> point, it should make it far easier to build a camlp4-aware js_of_ocaml
>>> toplevel, to fit into the experiments that OCamlPro is doing with web
>>> IDE interfaces.
>>>
>>> Anyway, these are still experiments and quite likely to run into unexpected
>>> issues as we build them, and we'll have to consider all the alternatives
>>> before folding them all into OPAM.
>>>
>>> Right now, OPAM dictates the *minimal* possible set of requirements for
>>> a package since it just uses shell commands.  It should be possible to
>>> specialise this for common cases such as a OASIS library, but it does
>>> require some careful thought first.  Luckily, we can use OPAM itself to
>>> conduct such experiments [1].
>>>
>>> All such discussions and experiments are very welcome on the mailing
>>> list (http://lists.ocaml.org/listinfo/opam-devel) to help establish
>>> everyone's use cases.  Daniel Buenzli's input has been particularly
>>> interesting to date, as his libraries adopt a very minimal shell-script
>>> driven approach that doesn't use OASIS, but still distributes META files.
>>> That's just one example of someone that doesn't use OASIS directly.
>>> The Core OASIS files are also a bit special due to their configuration
>>> requirements.  Xen/Mirage is also quite demanding in terms of the
>>> compilation environment.
>>>
>>> Here's my goal: I want to have near instant-rebuilds of Mirage packages.
>>> I know it's possible to pull this off with the existing compiler tools,
>>> since the monolithic Mirage tree (with lovingly crafted ocamlbuild rules
>>> and judicious use of native code compiler binaries) could rebuild the
>>> entire webserver in ~10s in parallel.  Nowadays, with OPAM, ocamlbuild,
>>> ocamlfind and camlp4, it takes around 2-3 minutes, which just isn't
>>> good enough.
>>>
>>> -anil
>>>
>>> [1] http://opam.ocamlpro.com/pkg/debug-camlp4-log.base.html: I hacked
>>> up a debug camlp4 to trace all the uses of camlp4 in the OPAM repository,
>>> during the wg-camlp4 discussion on the topic of where to go next with it.
>>> This just required doing a mass OPAM install, drinking some coffee, and
>>> reading the resulting trace logs.
>>>
>>> --
>>> Caml-list mailing list.  Subscription management and archives:
>>> https://sympa.inria.fr/sympa/arc/caml-list
>>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>>> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
> --
> Wojciech

Re: [Caml-list] GODI is shutting down

[Adrien Nader]

Hi,

I've been told on IRC that I had completely failed at explaining my
thoughts so I'll skip answering to your objections and try again.
Sorry for that, it was still a bit early and I didn't plan well the time
needed to proofread properly.

Oasis is completely independant of the package manager. It helps build
something that can be easily packaged but it doesn't go further, except
maybe by providing some helpers but that's a very small part of oasis.

It also setups the build system and opam doesn't do that. Opam only
works in your case because you've written a working build script but
that's not something easy and that's something that most usually failed.

It helps with the more complex builds, including C bindings. There were
mentions of it being an issue for bigger projects, I'm (geniously)
curious about them: what and how?

There's some overlap between oasis and your-build-scripts+opam but oasis
and opam simply aren't the same.

By the way, you should expect additional complexity soon as
cross-compilation becomes possible without additional patches to the
compiler. That's something a system like oasis could take care of.
I've cross-compiled libraries relying on oasis without issue by changing
ocamlfind's configuration. I'm under the impression oasis doesn't
differentiate yet between tools that need to run on the build and on the
host system (autotools parlance) but that would be easy even without
changing _oasis files with simple heuristics.

-- 
Adrien Nader

ocamlbuild (was Re: [Caml-list] Re: GODI is shutting down)

[Francois Berenger]

On 07/26/2013 05:03 AM, Adrien Nader wrote:
> Hi,
>
> I believe this entry on the bugtracker is worth mentionning:
>    "ocamlbuild should expose common interface as a library"
>    http://caml.inria.fr/mantis/view.php?id=5869
>
> More generally, ocamlbuild's development was mostly stalled but has
> restarted recently. This should allow fixing many issues and
> shortcomings.

There is not only ocamlbuild!

I would love to see projects such as obuild:

https://github.com/vincenthz/obuild

come out of the woods (like an official release announce if it is
ready for mass consumption) and become the de facto gold standard.

PS: even OCamlPro have written a tool to replace ocamlbuild
     (ocp-build)...

Re: ocamlbuild (was Re: [Caml-list] Re: GODI is shutting down)

[Peter Groves]

[-- Attachment #1: Type: text/plain, Size: 1730 bytes --]

As a shameless plug, I'll point out something I wrote several years ago
just for doing ad-hoc compilation (not building distributions), called
obake:

https://github.com/pgroves/obake

I use it often as it matches my workflow precisely. I doubt I'll put much
more work into it unless there is demand. The documentation is decent but
it has poor performance on large code bases.

It depends on ocamlfind already being configured to resolve dependencies.


On Thu, Jul 25, 2013 at 8:14 PM, Francois Berenger <berenger@riken.jp>wrote:

> On 07/26/2013 05:03 AM, Adrien Nader wrote:
>
>> Hi,
>>
>> I believe this entry on the bugtracker is worth mentionning:
>>    "ocamlbuild should expose common interface as a library"
>>    http://caml.inria.fr/mantis/**view.php?id=5869<http://caml.inria.fr/mantis/view.php?id=5869>
>>
>> More generally, ocamlbuild's development was mostly stalled but has
>> restarted recently. This should allow fixing many issues and
>> shortcomings.
>>
>
> There is not only ocamlbuild!
>
> I would love to see projects such as obuild:
>
> https://github.com/vincenthz/**obuild<https://github.com/vincenthz/obuild>
>
> come out of the woods (like an official release announce if it is
> ready for mass consumption) and become the de facto gold standard.
>
> PS: even OCamlPro have written a tool to replace ocamlbuild
>     (ocp-build)...
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/**arc/caml-list<https://sympa.inria.fr/sympa/arc/caml-list>
> Beginner's list: http://groups.yahoo.com/group/**ocaml_beginners<http://groups.yahoo.com/group/ocaml_beginners>
> Bug reports: http://caml.inria.fr/bin/caml-**bugs<http://caml.inria.fr/bin/caml-bugs>
>

[-- Attachment #2: Type: text/html, Size: 2609 bytes --]

Re: ocamlbuild (was Re: [Caml-list] Re: GODI is shutting down)

[Gabriel Scherer]

I am not very interested in build systems. I use one (ocamlbuild) that
I find decent (has several known areas of improvement, but bear with
me), and instead of rewriting a new one from scratch¹ I decided that
maybe the existing one I could be improved to fix its shortcomings.
You know, by writing bug reports, and more importantly proposing
patches (to the code or the documentation), instead of just
complaining on the mailing-list. I'm not aware, so far, of a major
design problem that can't be incrementally fixed, and whose resolution
is worth the pain of switching to a completely new system.

¹: or juggling with three different unannounced build systems; I think
you forgot Jenga and the various adaptations of redo for OCaml; nobody
added OCaml rules to Shake as far as I know, but certainly that will
happen in the future.

On Fri, Jul 26, 2013 at 3:14 AM, Francois Berenger <berenger@riken.jp> wrote:
> On 07/26/2013 05:03 AM, Adrien Nader wrote:
>>
>> Hi,
>>
>> I believe this entry on the bugtracker is worth mentionning:
>>    "ocamlbuild should expose common interface as a library"
>>    http://caml.inria.fr/mantis/view.php?id=5869
>>
>> More generally, ocamlbuild's development was mostly stalled but has
>> restarted recently. This should allow fixing many issues and
>> shortcomings.
>
>
> There is not only ocamlbuild!
>
> I would love to see projects such as obuild:
>
> https://github.com/vincenthz/obuild
>
> come out of the woods (like an official release announce if it is
> ready for mass consumption) and become the de facto gold standard.
>
> PS: even OCamlPro have written a tool to replace ocamlbuild
>     (ocp-build)...
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

[Caml-list] Re: ocamlbuild

[Francois Berenger]

On 07/26/2013 02:02 PM, Gabriel Scherer wrote:
> I am not very interested in build systems.

Me neither.
And, I'm not even competent enough to write one, unlike the many others
who felt they have to do so.

However, I do think (as a programmer) that the build system must
_NEVER_ get in front of my way.
It must be simple and observable (so that any problem
with the build configuration is easily fixed by anyone)
and parallelize the build to the maximum (this one should be a 
requirement from day 0 of the project).

> I use one (ocamlbuild) that
> I find decent (has several known areas of improvement, but bear with
> me), and instead of rewriting a new one from scratch¹ I decided that
> maybe the existing one I could be improved to fix its shortcomings.

At some point, I feel that some things have to be redesigned from
scratch rather than being patched endlessly.
Or, just switch to a better tool if it already exists.

> You know, by writing bug reports, and more importantly proposing
> patches (to the code or the documentation), instead of just
> complaining on the mailing-list.

You know, especially you Gabriel, that I can do that
(send patches with code, bug reports, etc.).
However, I do this only for projects that I admire.
Projects with a simple, crystal clear and sound design and a plain
UNIX taste.

> I'm not aware, so far, of a major
> design problem that can't be incrementally fixed, and whose resolution
> is worth the pain of switching to a completely new system.

I'm not sure there will be any pain for me to switch to another
build system (I think I will go for obuild).
I've stayed too long with ocamlbuild obviously.

Regards,
Francois.

Re: ocamlbuild (was Re: [Caml-list] Re: GODI is shutting down)

[Jeff Meister]

[-- Attachment #1: Type: text/plain, Size: 3468 bytes --]

I have a codebase that currently depends on an old version of OMake, and I
would rather use ocamlbuild instead. As an end user, I see two key
advantages: ocamlbuild is part of INRIA's official OCaml distribution,
which implies a certain level of commitment to continued support, and
ocamlbuild knows all about how to build OCaml projects (even using
ocamlfind), so for simple applications it "just works".

However, I was reluctant to look into ocamlbuild when it first appeared,
since the only documentation was a set of presentation slides. It seemed
powerful, but like you, I am not really interested in build systems. I'm
more encouraged now that a section has appeared in the OCaml manual. Aside
from that, are there any other resources I should look into? I'm wondering
if there are some documents, or even mailing list threads, that would help
me learn how to customize the building of a complex codebase like you just
did in the "which ocaml build system" thread? I'm hoping there is a way to
acquire this knowledge that doesn't involve trial and error with the build
system. :)


On Thu, Jul 25, 2013 at 10:02 PM, Gabriel Scherer <gabriel.scherer@gmail.com
> wrote:

> I am not very interested in build systems. I use one (ocamlbuild) that
> I find decent (has several known areas of improvement, but bear with
> me), and instead of rewriting a new one from scratch¹ I decided that
> maybe the existing one I could be improved to fix its shortcomings.
> You know, by writing bug reports, and more importantly proposing
> patches (to the code or the documentation), instead of just
> complaining on the mailing-list. I'm not aware, so far, of a major
> design problem that can't be incrementally fixed, and whose resolution
> is worth the pain of switching to a completely new system.
>
> ¹: or juggling with three different unannounced build systems; I think
> you forgot Jenga and the various adaptations of redo for OCaml; nobody
> added OCaml rules to Shake as far as I know, but certainly that will
> happen in the future.
>
> On Fri, Jul 26, 2013 at 3:14 AM, Francois Berenger <berenger@riken.jp>
> wrote:
> > On 07/26/2013 05:03 AM, Adrien Nader wrote:
> >>
> >> Hi,
> >>
> >> I believe this entry on the bugtracker is worth mentionning:
> >>    "ocamlbuild should expose common interface as a library"
> >>    http://caml.inria.fr/mantis/view.php?id=5869
> >>
> >> More generally, ocamlbuild's development was mostly stalled but has
> >> restarted recently. This should allow fixing many issues and
> >> shortcomings.
> >
> >
> > There is not only ocamlbuild!
> >
> > I would love to see projects such as obuild:
> >
> > https://github.com/vincenthz/obuild
> >
> > come out of the woods (like an official release announce if it is
> > ready for mass consumption) and become the de facto gold standard.
> >
> > PS: even OCamlPro have written a tool to replace ocamlbuild
> >     (ocp-build)...
> >
> >
> > --
> > Caml-list mailing list.  Subscription management and archives:
> > https://sympa.inria.fr/sympa/arc/caml-list
> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> > Bug reports: http://caml.inria.fr/bin/caml-bugs
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 4731 bytes --]

Re: ocamlbuild (was Re: [Caml-list] Re: GODI is shutting down)

[Gabriel Scherer]

> As an end user, I see two key
> advantages: ocamlbuild is part of INRIA's official OCaml distribution, which
> implies a certain level of commitment to continued support,

In all fairness, I should point out that this is not a good argument
in favor of OCamlbuild. While it is currently part of the official
distribution, it may be phased out in the future (the core maintainers
are eager to reduce the surface of code they have to maintain); as
some have mentioned it has seen disappointing levels of maintenance in
the past (since Nicolas Pouillard moved out to a post-doc, Xavier
Clerc did some work on ocamlbuild but has been very busy with other
things as well; only relatively recently did Wojciech put a fair
amount of effort into handling some of the users bugreports).

As I said above, I plan to keep using ocamlbuild and incrementally fix
some of its shortcomings (parallelization, documentation, plugin
composability...). I hope that some other users will also make a
contribution and help to solve some of the issues that have been
reported on the bugtracker (most of them are not very hard to tackle).
Wojciech and I eager to welcome patches and review them.

But I think you should pick your build system based on which one you
prefer (and you think can and will improve again in the long run),
rather than "political" aspects such as which one is blessed by this
or that organization. If you find out that some other build system
fits your needs better, by all means, use it.

> ocamlbuild
> knows all about how to build OCaml projects (even using ocamlfind), so for
> simple applications it "just works".

Yes, that's something I like with ocamlbuild. One corresponding
downside is that currently, the way things go wrong, when they go
wrong, is not very clear to users, so it may appear like a magic box
that works very well for simple projects, but does not degrade
carefully. I'm confident we can improve on that aspect in the future
-- and in practice I've found the _log file reasonably easy to
understand. But other people have made the choice to favor systems
that do less auto-detection, to make it easier to have a good
understanding of the build process.

> I'm more encouraged now that a section has appeared in the OCaml manual.

It's easy to improve the manual, so if you see something in this
section that can be improved, please provide a patch! (Or a bugreport,
possibly with a replacement/change suggestion; that's already very
helpful.)

> I'm wondering
> if there are some documents, or even mailing list threads, that would help
> me learn how to customize the building of a complex codebase like you just
> did in the "which ocaml build system" thread? I'm hoping there is a way to
> acquire this knowledge that doesn't involve trial and error with the build
> system. :)

Besides the ocamlbuild manual, the OCamlbuild wiki that I mentioned
before in this thread has helpful content:
  http://brion.inria.fr/gallium/index.php/Ocamlbuild
OCamlbuild also has some auto-documenting features that I mentioned in
this blog post:
  http://gallium.inria.fr/blog/quick-tip-the-ocamlbuild-documentation-option/

Besides that, I learned what I know about ocamlbuild by looking at
existing plugins, the _log, possibly with some level of verbosity, and
finally looking at the sources (admittedly not the preferred way to
documentation, but still better than "trial and error"), in particular
the ocaml_specific.ml file that encodes most of the build rules used
for OCaml programs.

On Fri, Jul 26, 2013 at 7:29 AM, Jeff Meister <nanaki@gmail.com> wrote:
> I have a codebase that currently depends on an old version of OMake, and I
> would rather use ocamlbuild instead. As an end user, I see two key
> advantages: ocamlbuild is part of INRIA's official OCaml distribution, which
> implies a certain level of commitment to continued support, and ocamlbuild
> knows all about how to build OCaml projects (even using ocamlfind), so for
> simple applications it "just works".
>
> However, I was reluctant to look into ocamlbuild when it first appeared,
> since the only documentation was a set of presentation slides. It seemed
> powerful, but like you, I am not really interested in build systems. I'm
> more encouraged now that a section has appeared in the OCaml manual. Aside
> from that, are there any other resources I should look into? I'm wondering
> if there are some documents, or even mailing list threads, that would help
> me learn how to customize the building of a complex codebase like you just
> did in the "which ocaml build system" thread? I'm hoping there is a way to
> acquire this knowledge that doesn't involve trial and error with the build
> system. :)
>
>
> On Thu, Jul 25, 2013 at 10:02 PM, Gabriel Scherer
> <gabriel.scherer@gmail.com> wrote:
>>
>> I am not very interested in build systems. I use one (ocamlbuild) that
>> I find decent (has several known areas of improvement, but bear with
>> me), and instead of rewriting a new one from scratch¹ I decided that
>> maybe the existing one I could be improved to fix its shortcomings.
>> You know, by writing bug reports, and more importantly proposing
>> patches (to the code or the documentation), instead of just
>> complaining on the mailing-list. I'm not aware, so far, of a major
>> design problem that can't be incrementally fixed, and whose resolution
>> is worth the pain of switching to a completely new system.
>>
>> ¹: or juggling with three different unannounced build systems; I think
>> you forgot Jenga and the various adaptations of redo for OCaml; nobody
>> added OCaml rules to Shake as far as I know, but certainly that will
>> happen in the future.
>>
>> On Fri, Jul 26, 2013 at 3:14 AM, Francois Berenger <berenger@riken.jp>
>> wrote:
>> > On 07/26/2013 05:03 AM, Adrien Nader wrote:
>> >>
>> >> Hi,
>> >>
>> >> I believe this entry on the bugtracker is worth mentionning:
>> >>    "ocamlbuild should expose common interface as a library"
>> >>    http://caml.inria.fr/mantis/view.php?id=5869
>> >>
>> >> More generally, ocamlbuild's development was mostly stalled but has
>> >> restarted recently. This should allow fixing many issues and
>> >> shortcomings.
>> >
>> >
>> > There is not only ocamlbuild!
>> >
>> > I would love to see projects such as obuild:
>> >
>> > https://github.com/vincenthz/obuild
>> >
>> > come out of the woods (like an official release announce if it is
>> > ready for mass consumption) and become the de facto gold standard.
>> >
>> > PS: even OCamlPro have written a tool to replace ocamlbuild
>> >     (ocp-build)...
>> >
>> >
>> > --
>> > Caml-list mailing list.  Subscription management and archives:
>> > https://sympa.inria.fr/sympa/arc/caml-list
>> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> > Bug reports: http://caml.inria.fr/bin/caml-bugs
>>
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>

Re: [Caml-list] Re: ocamlbuild

[Wojciech Meyer]

Francois Berenger <berenger@riken.jp> writes:

> At some point, I feel that some things have to be redesigned from
> scratch rather than being patched endlessly.

Please point out a single thing you would like to "redesign" in
ocamlbuild.

> However, I do this only for projects that I admire.
> Projects with a simple, crystal clear and sound design and a plain
> UNIX taste.

I believe ocamlbuild HAS clear and sound design, more than other
tools. In fact many ideas in ocamlbuild are novel, and interesting, this
is the major reason I maintain it, otherwise I'd not be interested in
doing so.

> I'm not sure there will be any pain for me to switch to another
> build system (I think I will go for obuild).
> I've stayed too long with ocamlbuild obviously.

Please do, and choose your build system (one of 6) which is best fit for you!

--
Wojciech

Re: [Caml-list] Re: ocamlbuild

[Francois Berenger]

On 07/26/2013 04:25 PM, Wojciech Meyer wrote:
> Francois Berenger <berenger@riken.jp> writes:
>
>> At some point, I feel that some things have to be redesigned from
>> scratch rather than being patched endlessly.
>
> Please point out a single thing you would like to "redesign" in
> ocamlbuild.

Here is one:
build configuration information scattered in several files of different 
syntax (myocamlbuild.ml and _tags).

>> However, I do this only for projects that I admire.
>> Projects with a simple, crystal clear and sound design and a plain
>> UNIX taste.
>
> I believe ocamlbuild HAS clear and sound design, more than other
> tools. In fact many ideas in ocamlbuild are novel, and interesting, this
> is the major reason I maintain it, otherwise I'd not be interested in
> doing so.
>
>> I'm not sure there will be any pain for me to switch to another
>> build system (I think I will go for obuild).
>> I've stayed too long with ocamlbuild obviously.
>
> Please do, and choose your build system (one of 6) which is best fit for you!
>
> --
> Wojciech
>

Re: [Caml-list] Re: ocamlbuild

[r.3]

[-- Attachment #1: Type: text/plain, Size: 1035 bytes --]

On 26/07/2013 10:07, Francois Berenger wrote: 

On 07/26/2013 04:25 PM, Wojciech Meyer wrote: 

<blockquote>
Francois Berenger <berenger@riken.jp> writes: 


<blockquote>
At some point, I feel that some things have to be redesigned from 
scratch rather than being patched endlessly. 



Please point out a single thing you would like to "redesign" in 
ocamlbuild. 

</blockquote>

Here is one: 
build configuration information scattered in several files of different syntax (myocamlbuild.ml and _tags). 

</blockquote>

If I am not mistaken, you can gather all information in myocamlbuild.ml. If not, that could be a bug report I guess, which seems possible to be solved. 

Personally, my projects are simple enough (as they only link to other ocaml packages using ocamlfind, without c dependencies), and I put everything in _tags file, no myocamlbuild.ml. On the contrary of gathering everything, for my purposes, I appreciate having a _tag file on each directory, adding up meta data to files located in the same place. 


William 


[-- Attachment #2: Type: text/html, Size: 1653 bytes --]

Re: [Caml-list] Re: ocamlbuild

[Daniel Bünzli]

Le vendredi, 26 juillet 2013 à 08:25, Wojciech Meyer a écrit :
> I believe ocamlbuild HAS clear and sound design, more than other
> tools. In fact many ideas in ocamlbuild are novel, and interesting, this
> is the major reason I maintain it, otherwise I'd not be interested in
> doing so.


I completely agree. When it came out I jumped on the project --- was burned out by a recursive make file trying to compile only things that changed across developing libraries used by another developing project; using just ocamlbuild, a _tags file and symlinks solved it all in a minute.

However:

* A wiki is not documentation. The reference document that documents the conceptual model of ocamlbuild, the available tags and their effects and a clear/understandable documentation of the *API* are still missing. How does ocamlbuild build your project, how does it compute dependencies, how does it track changes, etc.  

* Given our struggles with myocamlbuild.ml I wonder whether the current API is the right one or if it's just a matter of the missing documentation. And this is from a person who has written significant myocamlbuild.ml plugins [1,2]. If that is a sign of something my current strategy is to switch to a mixture of shell scripts and ocamlbuild invocations to side-step myocamlbuild.ml whenever possible.

* Easy built-in support for C stubs/libraries is missing. E.g. I'd be glad if I hadn't to write a myocamlbuild.ml file for a simple project like this [3].

Best,

Daniel

[1] http://brion.inria.fr/gallium/index.php/Compiling_C_with_gcc
[2] http://darcs.ocamlcore.org/ocamlunix/myocamlbuild.ml
[3] https://github.com/dbuenzli/rpng

Re: [Caml-list] Re: ocamlbuild

[Wojciech Meyer]

[-- Attachment #1: Type: text/plain, Size: 3635 bytes --]

> * A wiki is not documentation. The reference document that documents the
>   conceptual model of ocamlbuild, the available tags and their effects
>   and a clear/understandable documentation of the *API* are still
>   missing. How does ocamlbuild build your project, how does it compute
>   dependencies, how does it track changes, etc.

I agree we have to think, how to build up to date documentation as our
top priority. Embarrassingly it's been on my plate for quite long. Now I
think time is up, to update it.

> * Given our struggles with myocamlbuild.ml I wonder whether the current
>   API is the right one or if it's just a matter of the missing
>   documentation. And this is from a person who has written significant
>   myocamlbuild.ml plugins [1,2]. If that is a sign of something my
>   current strategy is to switch to a mixture of shell scripts and
>   ocamlbuild invocations to side-step myocamlbuild.ml whenever possible.

> * Easy built-in support for C stubs/libraries is missing. E.g. I'd be
>   glad if I hadn't to write a myocamlbuild.ml file for a simple project
>   like this [3].

The trouble is that OCamlbuild itself is really a build framework. It
contains a lot of rules out of the box, though, but they usually all
related to ocaml system, and not the external tools. Because we have no
method of distributing various plugins for OCamlbuild (Coq rules,
Js_of_ocaml rules, C rules, LaTeX rules etc.) it means that every
project hacks it own plugin, perhaps by copy pasting from the wiki. This is
un-satisfactory and a real root of the problem. Fortunately the solution
for this exist, we should allow ocamlfind packages to install needed
rules. We could say then:

ocamlbuild -use_package js_of_ocaml.ocamlbuild

Wojciech



On Fri, Jul 26, 2013 at 11:48 AM, Daniel Bünzli <daniel.buenzli@erratique.ch
> wrote:

> Le vendredi, 26 juillet 2013 à 08:25, Wojciech Meyer a écrit :
> > I believe ocamlbuild HAS clear and sound design, more than other
> > tools. In fact many ideas in ocamlbuild are novel, and interesting, this
> > is the major reason I maintain it, otherwise I'd not be interested in
> > doing so.
>
>
> I completely agree. When it came out I jumped on the project --- was
> burned out by a recursive make file trying to compile only things that
> changed across developing libraries used by another developing project;
> using just ocamlbuild, a _tags file and symlinks solved it all in a minute.
>
> However:
>
> * A wiki is not documentation. The reference document that documents the
> conceptual model of ocamlbuild, the available tags and their effects and a
> clear/understandable documentation of the *API* are still missing. How does
> ocamlbuild build your project, how does it compute dependencies, how does
> it track changes, etc.
>
> * Given our struggles with myocamlbuild.ml I wonder whether the current
> API is the right one or if it's just a matter of the missing documentation.
> And this is from a person who has written significant myocamlbuild.mlplugins [1,2]. If that is a sign of something my current strategy is to
> switch to a mixture of shell scripts and ocamlbuild invocations to
> side-step myocamlbuild.ml whenever possible.
>
> * Easy built-in support for C stubs/libraries is missing. E.g. I'd be glad
> if I hadn't to write a myocamlbuild.ml file for a simple project like
> this [3].
>
> Best,
>
> Daniel
>
> [1] http://brion.inria.fr/gallium/index.php/Compiling_C_with_gcc
> [2] http://darcs.ocamlcore.org/ocamlunix/myocamlbuild.ml
> [3] https://github.com/dbuenzli/rpng
>
>

[-- Attachment #2: Type: text/html, Size: 5028 bytes --]

Re: ocamlbuild (was Re: [Caml-list] Re: GODI is shutting down)

[Dario Teixeira]

Hi,

> As I said above, I plan to keep using ocamlbuild and incrementally fix

> some of its shortcomings (parallelization, documentation, plugin
> composability...). I hope that some other users will also make a
> contribution and help to solve some of the issues that have been
> reported on the bugtracker (most of them are not very hard to tackle).
> Wojciech and I eager to welcome patches and review them.

Please consider starting with the documentation, which is IMHO OCamlbuild's
Achilles's heel.  I will go as far as saying that what stopped OCamlbuild from
becoming the ultimate OCaml build tool and thus preempting the constant
popping up of new but often ephemeral build tools was precisely the lack
of good documentation.  Moreover, I reckon that even the other OCamlbuild
shortcomings (namely parallelisation) would have been patched long ago
if the tool had gained the community mind share that it missed due to its
poor documentation.

Daniel has already mentioned some of the things wrong with OCamlbuild's docs.
For me, the most egregious aspect of the manual is the lack of a good narrative
and a bird's eye view of the tool.  One can go through the whole document and
remain wondering how does it actually work and what does it do!  The reader
is thus left without a good mental model of the tool.  Moreover, bear in mind that
the wiki documentation is a good resource only *after* one already understands
how the tool works and what the various entities mean and how they fit together.

Anyway, I'm sure you (Gabriel and Wojciech) are well aware of the points I've just
raised, and time availability is the only limiting factor for improving the situation.
But since you've bravely "inherited" the maintenance of OCamlbuild, my 2 cents
is that your short term move that is likely to wield greater long term benefits is
to write a short introduction that allows the reader to really grok OCamlbuild.

Best regards,
Dario Teixeira

Re: [Caml-list] Re: ocamlbuild

[Ashish Agarwal]

[-- Attachment #1: Type: text/plain, Size: 360 bytes --]

On Fri, Jul 26, 2013 at 7:13 AM, Wojciech Meyer <wojciech.meyer@gmail.com>wrote:

I agree we have to think, how to build up to date documentation as our
> top priority.
>

I would be very happy if documenting ocamlbuild was prioritized over any
technical improvements. Additional features don't help me much since I
don't even understand the current features.

[-- Attachment #2: Type: text/html, Size: 711 bytes --]

Re: [Caml-list] Re: [Godi-list] GODI is shutting down

[Wojciech Meyer]

Daniel Bünzli <daniel.buenzli@erratique.ch> writes:

> Le mardi, 23 juillet 2013 à 16:32, Pierre-Etienne Meunier a écrit :
>> It seems that neither godi nor opam do any better (for this
> particular point, of course): the dictator is different in the three
> cases, but there is a dictator. In the case of opam, he has a friend
> dictator (github).
>
> I can't speak for godi, but at least for opam you get your facts
> wrong. There's nothing specific about github in opam and everybody can
> be a dictator. I'm one, just type:
>
>     opam repo add erratique-u http://erratique.ch/software/opam/unreleased
>     opam upgrade
>
>
> if you want to be one of my subjects.

Daniel,

Thanks for dictating how the excellent libraries should look like.

(and thanks for your libraries, gg is awesome!)

Best,
Wojciech