Re: [Caml-list] The DLL-hell of O'Caml

[Xavier Leroy <xavier.leroy@inria.fr>]

> Ugh.  I wholeheartedly disagree with source-only
> distribution.  Such a choice condemns ocaml and all of its
> applications to remain merely "hacker tools".  And all the
> worse to mandate a "all downstream dependencies recompile"
> policy.  I for one need to be able to someday hope that
> "regular users" will use my program.  Every ignorant user
> should _not_ be subjected to building and rebuild his own
> large system because of some deep semantics issue which
> researchers haven't quite cracked.

You're confusing whole programs with libraries.  A program can be
distributed as a statically-linked executable (ocamlopt -ccopt -static
or ocamlc -custom -ccopt -static), which will run happily on many,
many systems, and users will never have to worry about binary
compatibility and whatnot.  Only developers (= users of third-party
libraries) have to deal with code compatibility issues.

> Someone suggested the need of "closed source" as an argument
> for binary distribution. I can make an argument that relies
> on less.  Assume ala mathematical induction OCaml
> proliferates.  Foobar Corp does their bookkeeping in ocaml
> and has 2 million lines and needs to make a bugfix to their
> SQL wrapper way down at the bottom of the heap.  Do you tell
> them they must wait a couple of days do get their system
> back up?

This is a strawman argument in many respects:

- The OCaml distribution is 100 KLOC and recompiles in a few minutes
  on a 1000 EUR machine.  So, 2MLOC takes less than one hour, not two days.  
  Chances are that if you're developing a program of this size,
  you'll invest in a multiprocessor or compilation farm, bringing
  the rebuild time even lower.  (Say, just long enough to have coffee,
  chat with your coworkers, and check Slashdot.  Isn't programmer's life
  sweet? :-)

- If I were to design an application of this size, I'll try very hard
  not to make it into one executable, but rather decompose it into
  several executables communicating via IPC.  Hence, only the parts
  that use the SQL wrapper have to be recompiled.

- Even if the program consists of one executable, it is easy to recompile
  only the modules that really depend on the SQL library: instead of doing
  "make clean; make all"; just generate dependencies with
  ocamldep -I/the/path/to/the/SQL/library and do "make".
  That we generally don't bother to record dependencies on
  third-party libraries in Makefiles doesn't mean it can't be done 
  if need arises.

> Imagine if the computer on your desk was built out of a (C
> based) source distro.  Not just a source distro, but one
> with the "all downstream dependencies recompile" policy.

This is not what I'm proposing.  It is perfectly feasible to make a
binary OCaml "distribution" (in the sense of Debian and RedHat being
"distributions") such as the CDK.  Someone (the distributor) just did
all the work of building a large collection of sources for you.

Indeed, a truly automated build procedure like I propose would make
the work of the CDK maintainers much easier.

What I think is futile is to expect users to pick independently-
compiled binary distributions of libraries and dynamically-linked executables
left and right, and expect that the resulting system will work.  Try
to do this with a Linux system and it will break very quickly.

> (I don't think anybody has been so deluded to suggest this in a
> large system.)  Now suppose you write device drivers.
> Device driver programmers were frustrated enough before the
> linux kernel was modularized so that they could load and
> unload them without relinking the kernel and rebooting.  You
> mean to tell them now that they must recompile not only the
> kernel, but therefore the entire machine every time they
> make a one line bugfix?  What kind of use would a machine
> like that be?

More rethorical questions that don't make much sense to me.
Loading/unloading kernel modules during kernel development is at best
a dynamic loading issue.  That they might have to recompile their
driver when the kernel changes doesn't bother kernel developers in the
least.

> As a historical reference, the Debian project, having the
> hacker mentality they do, apparently also fanatasized about
                                            ^^^^^^^^^^^
I like this new word :-)

> having a source only distribution (of course I don't imagine
> it was an "all downstream dependencies recompile" design):
> 
> >"At the very early stages of the Project, members
> considered
> >distributing source-only packages.  Each package would
> consist
> >of the upstream source code and a Debianized patch file,
> and users
> >would untar the sources, apply the patches, and compile
> binaries
> >themselves.  They soon realized, however, that some sort of
> >binary distribution scheme would be needed.  The earliest
> packaging
> >tool, written by Ian Murdock and called dpkg, created a
> package in
> >a Debian-specific binary format, and could be used later to
> unpack
> >and install the files in the package."

Again, I don't have anything against binary distributions, as long as
there's an underlying source distribution and recompilation manager
that lets me bring things up to date if a binary incompatibility
arises.  RPMs get close to this but lack a "rebuild and install what
needs to be recompiled" command.  Maybe Debian's packages or BSD ports
are better in this respect.

> To change the subject slightly, there seem to be some other
> dangerous related "purity before progress" idea bugs
> floating around, like

And there seems to be some dangerous "let's hack something before
thinking over the consequences" ideas in your message.

- Xavier Leroy
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners