Re: Problems to get larger user base ... (Re: [Caml-list] OCaml's variables)

[Malcolm Matalka <mmatalka@gmail.com>]

No I'm not suggesting building on every machine.  I am suggesting using
a Build Server, such as Jenkins or Bamboo or Hydra, to build your system
and package it, then you can deploy it through another mechanism.

Opam generating RPMs (or packages for you) I believe is wrong because:

1 - There are a lot of options when it comes to building a package.
What should the init scripts look like?  Where should the various files
go?  What about configs, etc.  These are issues opam should not concern
it self with.  On top of that, there are already tools for building a
RPM packages from a directory structure, such as fpm.

2 - Which package systems should Opam support?  RPM, Deb?  I use Nix,
why don't you support me?  Some other guy uses something else, what
about him?  I would predict a lot of heated and ultimately fruitless ML
discussions about people wanting their package support in a particular way.

3 - *Maybe* Opam should support exporting the closure of a package to an
external directory structure, then you can package it in any way you
want.

However, IME, and someone with more knowledge in Ocaml please correct
me, if you are deploying to production on a lot of machines I don't
really see why you would want to install Ocaml libraries to those
machines.  Ocaml libraries are statically linked so copying the produced
binary should be sufficient.  This is how I deploy Ocaml apps to the
clusters I have worked on, with success.

Perhaps I am missing something?

If you want a 'deploy' command to be added to Opam that takes a
configuration and pushes builds to machines, I am vehemently opposed to
that.  If you want a mechanism for exporting a package closure someplace
else that you can package, I am less opposed to that.  But I don't
develop Opam so my opinion doesn't really matter :)

/M

Chet Murthy <murthy.chet@gmail.com> writes:

> I'm glad we're having this discussion.
>
> On Thursday, May 30, 2013 06:52:25 AM Malcolm Matalka wrote:
>> I think out would be wrong for opam to try to solve this problem.  There
>> are already many tools available for deploying (Ansible, Puppet, Chef,
>> Fabric, Capistrano).  Such a later can be build on top of opam of need be.
>
> I think this is incorrect.  Let me explain.
>
> (1) when we look at deploying complex collections of code/libs/data
> onto multiple machines, usually we assume that the code has already
> been built.
>
> (2) but let's first dispatch the case where the code has -not- been
> built.  In such a case, I presume you're proposing that the code be
> built on each machine, yes?
>
>   (a) this drastically increases the CPU required to perform upgrades
>   and deploys
>
>   (b) but far, far, far more importantly, it means that on each
>   machine, a nontrivially complex script runs that builds the actual
>   installed binaries.  If that script contains -any- nondeterminism or
>   environmental sensitivity, it could produce different results on
>   different machines.  The technical term is "version skew".
>
> In scale-out systems, this sort of "skew" is absolutely fatal, because
> it means that machines/nodes are not a priori interchangeable.  And
> all of fast-fail fault-tolerance depends on nodes being
> interchangeable.
>
> (3) But let's say that what you really mean is that we should use
> tools like puppet/chef/capistrano to copy collections of
> binaries/libs/data to target machines and install them.  These
> scripts/recipes are written by some person.  You could have equally
> well suggested that that person build Debian packages (or RPMs) of
> each OPAM package, writing out all the descriptions and manifests.
>
> And manually specifying all dependencies and requiremeents.
>
> Either way, that person is doing a job that OPAM already does a lot
> of, and does quite well.  Gosh, wouldn't it be nice if OPAM could
> generate those RPMs?  Well, it's a little more complicated than that,
> but really, not much more.  The complexity comes in that you -might-
> (I'm not saying I have this part figured out yet) want ways to
> -generalize- (say) the camlp5 package so that it could be installed on
> many different base OPAM installations.
>
> But setting aside that nice-to-have, imagine that OPAM knew how to
> generate RPMs from each package it installed, and from the ocaml+opam
> base itself.  You combine those, and you can:
>
>   (i) install ocaml, opam, and a bunch of packages
>
>   (ii) push a button, and out come a pile of RPMs, along with
>   dependencies amongst them (and hopefully on the relevant
>   environmental RPMs (e.g., libpcre-dev for pcre-ocaml, etc) so that
>   you can just stuff those RPMs into a YUM repo, go to a second box,
>   and say
>
>     "yum install opam ocaml pcre-ocaml"
>
>   and get everything slurped down and installed, just as if OPAM had
>   installed it all, package-by-package.
>
> --chet--
>
> P.S. And this doesn't even get into the unsuitability of chef/puppet
> for managing software package installation.  There's a reason that no
> distro uses such schemes to install the large and complex sets of
> packages needed to run amodern Linux box.  And why there is no Linux
> version of Microsorft's "DLL Hell".  Linux distros by and large (and
> esp Debian and Ubuntu) have worked hard to make package installation
> foolproof -- and chef/puppet etc are anything but.