Re: [Caml-list] Big executables from ocamlopt; dynamic libraries again

[Jacques Garrigue <garrigue@kurims.kyoto-u.ac.jp>]

From: tim@fungible.com (Tim Freeman)

> >A major problem with using dynamic linking with ocaml (in particular
> >with native code), is that your program come cut into small pieces,
> >and you must be sure that they are all compatible.
> 
> How is ocaml different from C in this regard?  One difference is that
> ocaml is younger and therefore changing faster, but eventually that
> won't be true any more.  Are there other difference?

In short: C doesn't make sure that they are compatible.
If they are, this will work, otherwise, undefined behaviour.
Programmers and users are responsible for checking (by hand!) that the
API didn't change in an incompatible way.

If you want to have both security and allow linking everytime it's
safe, then you would need to do lots of type-checking at link-time
(runtime for dynamic linking). Basically that every module you depend
on has an interface at least as good as what you need, checking type
by type. If you've got a look at the size of some .cmi's, you may
realize that including required types in executables may require
potentially huge sizes. And type-checking is sometimes too slow.

As a fall-back solution, there is MD5 hashing. The problem is that
you're then mixing information for all the contents of a module.
Any change will produce a new incompatible hash value.
For instance, every time you add a function to a library, it becomes
incompatible. And there are new functions in every release of ocaml.

Note that for C, compatibility policies generally allow adding extra
functions to a library without changing the version number, since the
problem, should it arise, can be detected at link time.

And, even worse than that, the current MD5 computation scheme is
algorithm dependent: it is not based on a normalized view of types,
but just on a dump of an internal tree structure, which is extremely
sensitive to any change in the type checking algorithm. This means
that compatibility can be broken as often as once a week for the CVS
version!
I suppose one could define specific normalizing picking and unpicking
procedures, rather than using output_value and input_value as
currently, but this would be a fair amount of work, and I'm not even
sure it would solve completely the problem.

> >And this nullifies the zlib example: the newly compiled version would
> >not be compatible with existing executables!
> 
> Is there something wrong with just checksumming the signature to
> decide whether the code is compatible?  That would still be more
> sensitive than I'd like, since adding to the signature ideally would
> not require people using the package to recompile, but it ought to
> support the zlib example.

The main proablem being with incompatibilities between different
version of ocaml, any code compiled with ocaml cannot be given
compatible MD5 signatures... So, yes, your zlib example would work,
but only for the 6 months between two releases of ocaml.
This can be OK if you have a fair control of what you are running, but
this would be nightmare for the average user.


So probably the real answer is that dynamic linking of caml native
code is possible, but that it would be a lot of work, not so much at
the compilation level, but more at improving compatibility checking.
One could argue that the benefits would not be limited to dynamic
linking alone, but also easier upgrading bewteen ocaml versions, so
this might be worth it.

Cheers,

        Jacques Garrigue
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners