RE: [Caml-list] The DLL-hell of O'Caml

["Jeff Henrikson" <jehenrik@yahoo.com>]

> Conclusion for OCaml?  Forget about binary compatibility,
binary-only
> distributions, versioning, and all that.  All we need for
OCaml
> libraries is source distribution, a standardized
re-building procedure,
> and a place to record cross-library dependencies, so that
everything
> that needs to be recompiled when something changes can be
recompiled
> automatically.  Something like a source RPM or Debian
package, or a
> BSD port.

Ugh.  I wholeheartedly disagree with source-only
distribution.  Such a choice condemns ocaml and all of its
applications to remain merely "hacker tools".  And all the
worse to mandate a "all downstream dependencies recompile"
policy.  I for one need to be able to someday hope that
"regular users" will use my program.  Every ignorant user
should _not_ be subjected to building and rebuild his own
large system because of some deep semantics issue which
researchers haven't quite cracked.  No matter how automated
recompilation could be, the (potentially repeated) time
penalty involved would be unacceptable.

Someone suggested the need of "closed source" as an argument
for binary distribution.  I can make an argument that relies
on less.  Assume ala mathematical induction OCaml
proliferates.  Foobar Corp does their bookkeeping in ocaml
and has 2 million lines and needs to make a bugfix to their
SQL wrapper way down at the bottom of the heap.  Do you tell
them they must wait a couple of days do get their system
back up?  Suppose they did one trial run and they find out
they put the wrong function call in.  Two more days, ad
nausem.

Imagine if the computer on your desk was built out of a (C
based) source distro.  Not just a source distro, but one
with the "all downstream dependencies recompile" policy.  (I
don't think anybody has been so deluded to suggest this in a
large system.)  Now suppose you write device drivers.
Device driver programmers were frustrated enough before the
linux kernel was modularized so that they could load and
unload them without relinking the kernel and rebooting.  You
mean to tell them now that they must recompile not only the
kernel, but therefore the entire machine every time they
make a one line bugfix?  What kind of use would a machine
like that be?

As a historical reference, the Debian project, having the
hacker mentality they do, apparently also fanatasized about
having a source only distribution (of course I don't imagine
it was an "all downstream dependencies recompile" design):

>"At the very early stages of the Project, members
considered
>distributing source-only packages.  Each package would
consist
>of the upstream source code and a Debianized patch file,
and users
>would untar the sources, apply the patches, and compile
binaries
>themselves.  They soon realized, however, that some sort of
>binary distribution scheme would be needed.  The earliest
packaging
>tool, written by Ian Murdock and called dpkg, created a
package in
>a Debian-specific binary format, and could be used later to
unpack
>and install the files in the package."

from:
http://www.debian.org/doc/manuals/project-history/ch4.html


To change the subject slightly, there seem to be some other
dangerous related "purity before progress" idea bugs
floating around, like

garrigue@kurims.kyoto-u.ac.jp wrote:

> The real problem is about how to check that binary (and
> semantics) compatibility is satisfied.

Exactly what sort of semantics?  Certainly the general case
is turing-complete.  What reasonable subset could you mean
by this?  And just why do we care?  Nobody's pretending type
checking makes your program correct.


>>>>I don't see this.  I can believe that consequences of
implementation
>>>>choices which have been made prohibit this.  For
hypothetical example,
>>>>inlining behavior which could not be disabled on public
interfaces would
>>>>be a problem.  (I don't think this particular thing
happens in ocaml.)
>>>>But I certainly don't see "a consequence of strict
typing."
>>>>
>>
>> I agree.  It is not true of other languages with static t
yping,
>> e.g. Mercury or C++.
>
>I don't know Mercury but C++. Actually, it does not ensure
proper
>typing between compilation units! The .o files do not
contain types,
>and it is up to the programmer to include the right
prototypes.

Yes, it's called using the right header files.  That and
version numbering your runtime libraries.  Just like the
rest of the computing world.   Apparently forced type
checking is an open problem, so lets not sit and cry over
it.  Fortunately it hasn't been solved anywhere else, so it
is a universally understood programmer's convention to
couple libraries with source code interfaces.  Are we really
so special that we must break it?

This is the scenario I envision.  I don't think the design
choices above are hard, though there still may be some
substantial work because apparently a mutually exclusive set
of assumptions has governed the evolution of the compiler up
until this point.

	- The compiler must guarentee a fixed calling convention
and representation for all concrete types for a given .mli
file.
	- Types parameterized by abstract types (eg "list", sorry
if my jargon is ignorant) must not change their compilation
based on knowledge of a new type.
	- Functors must have the analogous property
	- If the above is not generally runtime efficient, then it
must be made so by providing a "public" or "external" or
some such keyword to allow the interfaces which are exposed
to have these properties.

(corrections/elaborations to the above are welcome feedback)

There _do_ currently exist design choices in the ocaml
sources that arbitrarily for no reason violate this.  One of
them was fixed for SCaml, which was that arbitrary gensymed
numbers were being added to external function definitions
from ocamlopt.  Eg I implement:

foo.ml
	let f x = x + 1;;

and the compiler generates a .so symbol called "f17,"
whereas

foo.ml
	let f x = x + x + 1;;

might generate "f23" or some other arbitrary thing.  Merely
changing the length of procedure bodies can make a different
number of gensym requests happen and therefore the number 17
morphs into something else.  Not good for linking against.
The SCaml patch does fix this particular thing, but there
are likely to be others.

I reiterate that I haven't seen a concrete counterexample
which demonstrates that the semantics themselves make
runtime interfaces impossible.  A previous post of mine
answered with one counterexample which changed optional
arguments across a public interface.  That won't work in C++
or any other of the myriad languages which do have
deterministic runtime interfaces.  Apologies that I have not
read in detail a couple of examples using functors that came
in yesterday, but I had to finish this email sometime.

I additionally point out that on course granularity, dynamic
typing ala COM interface IDs and class IDs works pretty
well.  (These are just large 128 bit numbers based on things
like the creator's ethernet hardware address and a fine
granularity time stamp, designed to be a unique type code
for every possible interface and class).  The systems which
don't have the right library versions at least crash right
away when the module is linked rather than run for a while
and segfault when a function is utilized.  It's not perfect,
but its ubiquity demonstrates that it works in practice.

In closing, I'd like to ask a more tenuous and subjective
question.  Is there any fundamental reason why the caml
package manager can't _be_ the Debian package manager?  I
think this would be a big step in the maturity and ubiquity
of the OCaml community at large.  And I mean dependencies
_both_ ways, C calling Caml as well as Caml calling C and
other Caml.  (Not that people these days are publishing a
lot of C interfaces for their Caml libraries.  To integrate
with other programmer communities that should change but
it's another issue.)  I haven't thought about this one a
lot, but having a quirky on-the-side package manager for a
special quirky language gives to me the general impression
of obscurity by design.


Jeff Henrikson




-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners