From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 62D0E7EE51 for ; Thu, 30 May 2013 08:02:24 +0200 (CEST) Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of mmatalka@gmail.com) identity=pra; client-ip=74.125.82.41; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="mmatalka@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of mmatalka@gmail.com designates 74.125.82.41 as permitted sender) identity=mailfrom; client-ip=74.125.82.41; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="mmatalka@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-wg0-f41.google.com) identity=helo; client-ip=74.125.82.41; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="mmatalka@gmail.com"; x-sender="postmaster@mail-wg0-f41.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqQBAFzqplFKfVIpjWdsb2JhbABZgmjCWoECFg4BAQEBBwsLCRIGJIIjAQEEAUABGx0BAwELBgULDQklDwEEDQIRAQUBIhOHegEDCQYBnBmMP4J9hG0KGScNWIgiAQUMjDCBMoElB4NXA5VVgWaMHYM+P4Q1gWo X-IPAS-Result: AqQBAFzqplFKfVIpjWdsb2JhbABZgmjCWoECFg4BAQEBBwsLCRIGJIIjAQEEAUABGx0BAwELBgULDQklDwEEDQIRAQUBIhOHegEDCQYBnBmMP4J9hG0KGScNWIgiAQUMjDCBMoElB4NXA5VVgWaMHYM+P4Q1gWo X-IronPort-AV: E=Sophos;i="4.87,768,1363129200"; d="scan'208";a="16147083" Received: from mail-wg0-f41.google.com ([74.125.82.41]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 30 May 2013 08:02:23 +0200 Received: by mail-wg0-f41.google.com with SMTP id k13so4733212wgh.4 for ; Wed, 29 May 2013 23:02:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-type; bh=5CpDW6NP3pcy2Z/7yHahdQVAoGznRxmAZa7dQ9N2D3E=; b=chKxwxL/xXODHb8qqYs5DIVEBYm3LupbVaKVn3Z+7JfAUCxBVyH6efWFlrMOx+eWF0 hrKxD+tA0/aH6XPdCE1ezmvuTw6CDwY9jLEY4pjjM0f4UBFISwXaVTg091n3qziWeXI6 i4BenA5gCg+s+giLaAEOeQ91dk46+Ex8YVEDFpFbjar1D2Z3BLxmxoij1X3JoNi5kyqT d/ld4aOufAiF9uKisLyAkZueB+pJ8fIxpHnMFA+TDtAlnAk+S4rJiLLLwhPpLp9k6EDK RhYAzyoXUAgi2zlDxHblNYQEd6mqJoLSKkziCE7USvVVg9LEBXcWsP3/ER57+1OmrrVw wI+g== X-Received: by 10.180.208.81 with SMTP id mc17mr17060355wic.35.1369893743541; Wed, 29 May 2013 23:02:23 -0700 (PDT) Received: from localhost ([2a01:7e00::f03c:91ff:fe70:2696]) by mx.google.com with ESMTPSA id d10sm35580992wik.0.2013.05.29.23.02.22 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Wed, 29 May 2013 23:02:22 -0700 (PDT) From: Malcolm Matalka To: Chet Murthy Cc: Francois Berenger , caml-list References: <20130523235355.GI6510@siouxsie> <1629005.lH05byJ9SH@groupon> <1804446.xtBoISCFl2@groupon> Date: Thu, 30 May 2013 06:02:22 +0000 In-Reply-To: <1804446.xtBoISCFl2@groupon> (Chet Murthy's message of "Wed, 29 May 2013 22:05:15 -0700") Message-ID: <87a9ndovip.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Subject: Re: Problems to get larger user base ... (Re: [Caml-list] OCaml's variables) No I'm not suggesting building on every machine. I am suggesting using a Build Server, such as Jenkins or Bamboo or Hydra, to build your system and package it, then you can deploy it through another mechanism. Opam generating RPMs (or packages for you) I believe is wrong because: 1 - There are a lot of options when it comes to building a package. What should the init scripts look like? Where should the various files go? What about configs, etc. These are issues opam should not concern it self with. On top of that, there are already tools for building a RPM packages from a directory structure, such as fpm. 2 - Which package systems should Opam support? RPM, Deb? I use Nix, why don't you support me? Some other guy uses something else, what about him? I would predict a lot of heated and ultimately fruitless ML discussions about people wanting their package support in a particular way. 3 - *Maybe* Opam should support exporting the closure of a package to an external directory structure, then you can package it in any way you want. However, IME, and someone with more knowledge in Ocaml please correct me, if you are deploying to production on a lot of machines I don't really see why you would want to install Ocaml libraries to those machines. Ocaml libraries are statically linked so copying the produced binary should be sufficient. This is how I deploy Ocaml apps to the clusters I have worked on, with success. Perhaps I am missing something? If you want a 'deploy' command to be added to Opam that takes a configuration and pushes builds to machines, I am vehemently opposed to that. If you want a mechanism for exporting a package closure someplace else that you can package, I am less opposed to that. But I don't develop Opam so my opinion doesn't really matter :) /M Chet Murthy writes: > I'm glad we're having this discussion. > > On Thursday, May 30, 2013 06:52:25 AM Malcolm Matalka wrote: >> I think out would be wrong for opam to try to solve this problem. There >> are already many tools available for deploying (Ansible, Puppet, Chef, >> Fabric, Capistrano). Such a later can be build on top of opam of need be. > > I think this is incorrect. Let me explain. > > (1) when we look at deploying complex collections of code/libs/data > onto multiple machines, usually we assume that the code has already > been built. > > (2) but let's first dispatch the case where the code has -not- been > built. In such a case, I presume you're proposing that the code be > built on each machine, yes? > > (a) this drastically increases the CPU required to perform upgrades > and deploys > > (b) but far, far, far more importantly, it means that on each > machine, a nontrivially complex script runs that builds the actual > installed binaries. If that script contains -any- nondeterminism or > environmental sensitivity, it could produce different results on > different machines. The technical term is "version skew". > > In scale-out systems, this sort of "skew" is absolutely fatal, because > it means that machines/nodes are not a priori interchangeable. And > all of fast-fail fault-tolerance depends on nodes being > interchangeable. > > (3) But let's say that what you really mean is that we should use > tools like puppet/chef/capistrano to copy collections of > binaries/libs/data to target machines and install them. These > scripts/recipes are written by some person. You could have equally > well suggested that that person build Debian packages (or RPMs) of > each OPAM package, writing out all the descriptions and manifests. > > And manually specifying all dependencies and requiremeents. > > Either way, that person is doing a job that OPAM already does a lot > of, and does quite well. Gosh, wouldn't it be nice if OPAM could > generate those RPMs? Well, it's a little more complicated than that, > but really, not much more. The complexity comes in that you -might- > (I'm not saying I have this part figured out yet) want ways to > -generalize- (say) the camlp5 package so that it could be installed on > many different base OPAM installations. > > But setting aside that nice-to-have, imagine that OPAM knew how to > generate RPMs from each package it installed, and from the ocaml+opam > base itself. You combine those, and you can: > > (i) install ocaml, opam, and a bunch of packages > > (ii) push a button, and out come a pile of RPMs, along with > dependencies amongst them (and hopefully on the relevant > environmental RPMs (e.g., libpcre-dev for pcre-ocaml, etc) so that > you can just stuff those RPMs into a YUM repo, go to a second box, > and say > > "yum install opam ocaml pcre-ocaml" > > and get everything slurped down and installed, just as if OPAM had > installed it all, package-by-package. > > --chet-- > > P.S. And this doesn't even get into the unsuitability of chef/puppet > for managing software package installation. There's a reason that no > distro uses such schemes to install the large and complex sets of > packages needed to run amodern Linux box. And why there is no Linux > version of Microsorft's "DLL Hell". Linux distros by and large (and > esp Debian and Ubuntu) have worked hard to make package installation > foolproof -- and chef/puppet etc are anything but.