* Re: [Caml-list] The DLL-hell of O'Caml
2002-03-20 20:39 ` Xavier Leroy
@ 2002-03-20 21:16 ` Markus Mottl
2002-03-21 9:07 ` Warp
` (3 subsequent siblings)
4 siblings, 0 replies; 26+ messages in thread
From: Markus Mottl @ 2002-03-20 21:16 UTC (permalink / raw)
To: Xavier Leroy; +Cc: Gerd Stolpmann, Johan Georg Granström, caml-list
On Wed, 20 Mar 2002, Xavier Leroy wrote:
> All we need for OCaml libraries is source distribution, a standardized
> re-building procedure,
Yes, please!
> I believe this is a much more reasonable objective than dreaming
> about binary compatibility and whatnot. Still, it's not trivial:
> the authors of existing OCaml libraries (myself included) have some
> work to do to implement fully automatic build of their libraries...
Good to know that the INRIA-team is not neglecting this burning issue
and thinking about solutions. I really hope that OCaml libraries get the
reverse transcriptase they need to become really virulent for spreading
to all machines on this planet :-)
Regards,
Markus Mottl
--
Markus Mottl markus@oefai.at
Austrian Research Institute
for Artificial Intelligence http://www.oefai.at/~markus
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Caml-list] The DLL-hell of O'Caml
2002-03-20 20:39 ` Xavier Leroy
2002-03-20 21:16 ` Markus Mottl
@ 2002-03-21 9:07 ` Warp
2002-03-21 10:18 ` Christopher Quinn
` (2 subsequent siblings)
4 siblings, 0 replies; 26+ messages in thread
From: Warp @ 2002-03-21 9:07 UTC (permalink / raw)
To: OCaml; +Cc: caml-list
> Conclusion for OCaml? Forget about binary compatibility, binary-only
> distributions, versioning, and all that. All we need for OCaml
> libraries is source distribution, a standardized re-building procedure,
> and a place to record cross-library dependencies, so that everything
> that needs to be recompiled when something changes can be recompiled
> automatically. Something like a source RPM or Debian package, or a
> BSD port.
So, do we need to forget about any commercial usage of OCaml ?
In fact, commercial usage needs real "closed-source" libraries where only
bytecode files are distributed. Such libraries should be able to evolve , by
adding new functions into the interface, and to be loaded with signature
checking at run-time. Does the OCaml team don't plan at all to add such
feature ?
Nicolas Cannasse
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Caml-list] The DLL-hell of O'Caml
2002-03-20 20:39 ` Xavier Leroy
2002-03-20 21:16 ` Markus Mottl
2002-03-21 9:07 ` Warp
@ 2002-03-21 10:18 ` Christopher Quinn
2002-03-21 18:13 ` Xavier Leroy
2002-03-21 14:13 ` Jeff Henrikson
2002-03-21 18:50 ` Sven
4 siblings, 1 reply; 26+ messages in thread
From: Christopher Quinn @ 2002-03-21 10:18 UTC (permalink / raw)
To: Xavier Leroy; +Cc: caml-list
>
> But A.cmo does not contain SIGB -- only a mention of B and a hash of
> SIGB. The reason is quite simple: A.cmo would be *huge* if it
> included a copy of the signatures of every module it refers, either
> directly or indirectly. (Think several megabytes.)
Any idea how much improvement would come from hash-consing,
as Dave Berry suggested? And only that part of a module's
signature which pertained to actual usage would need
inclusion, no? And only of those modules externally provided
upon which one's project depends, ie. the standard library,
not the internal project modules?
Inclusion of such signature fragments need only be an
optional feature when building distributed libraries, not
generally for development (inhabiting every .cmo), so the
'bloat' is at least confined to where it matters, at the
boundary.
Chris Q.
be an optional feature of libraries alone, thus confining
any sort of bloat to distributed libs, commercial or otherwise?
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
^ permalink raw reply [flat|nested] 26+ messages in thread
* RE: [Caml-list] The DLL-hell of O'Caml
2002-03-20 20:39 ` Xavier Leroy
` (2 preceding siblings ...)
2002-03-21 10:18 ` Christopher Quinn
@ 2002-03-21 14:13 ` Jeff Henrikson
2002-03-21 14:13 ` [Caml-list] Type-safe DLL's with OO (was DLL-hell of O'Caml) Tim Freeman
2002-03-21 18:10 ` [Caml-list] The DLL-hell of O'Caml Xavier Leroy
2002-03-21 18:50 ` Sven
4 siblings, 2 replies; 26+ messages in thread
From: Jeff Henrikson @ 2002-03-21 14:13 UTC (permalink / raw)
To: caml-list
> Conclusion for OCaml? Forget about binary compatibility,
binary-only
> distributions, versioning, and all that. All we need for
OCaml
> libraries is source distribution, a standardized
re-building procedure,
> and a place to record cross-library dependencies, so that
everything
> that needs to be recompiled when something changes can be
recompiled
> automatically. Something like a source RPM or Debian
package, or a
> BSD port.
Ugh. I wholeheartedly disagree with source-only
distribution. Such a choice condemns ocaml and all of its
applications to remain merely "hacker tools". And all the
worse to mandate a "all downstream dependencies recompile"
policy. I for one need to be able to someday hope that
"regular users" will use my program. Every ignorant user
should _not_ be subjected to building and rebuild his own
large system because of some deep semantics issue which
researchers haven't quite cracked. No matter how automated
recompilation could be, the (potentially repeated) time
penalty involved would be unacceptable.
Someone suggested the need of "closed source" as an argument
for binary distribution. I can make an argument that relies
on less. Assume ala mathematical induction OCaml
proliferates. Foobar Corp does their bookkeeping in ocaml
and has 2 million lines and needs to make a bugfix to their
SQL wrapper way down at the bottom of the heap. Do you tell
them they must wait a couple of days do get their system
back up? Suppose they did one trial run and they find out
they put the wrong function call in. Two more days, ad
nausem.
Imagine if the computer on your desk was built out of a (C
based) source distro. Not just a source distro, but one
with the "all downstream dependencies recompile" policy. (I
don't think anybody has been so deluded to suggest this in a
large system.) Now suppose you write device drivers.
Device driver programmers were frustrated enough before the
linux kernel was modularized so that they could load and
unload them without relinking the kernel and rebooting. You
mean to tell them now that they must recompile not only the
kernel, but therefore the entire machine every time they
make a one line bugfix? What kind of use would a machine
like that be?
As a historical reference, the Debian project, having the
hacker mentality they do, apparently also fanatasized about
having a source only distribution (of course I don't imagine
it was an "all downstream dependencies recompile" design):
>"At the very early stages of the Project, members
considered
>distributing source-only packages. Each package would
consist
>of the upstream source code and a Debianized patch file,
and users
>would untar the sources, apply the patches, and compile
binaries
>themselves. They soon realized, however, that some sort of
>binary distribution scheme would be needed. The earliest
packaging
>tool, written by Ian Murdock and called dpkg, created a
package in
>a Debian-specific binary format, and could be used later to
unpack
>and install the files in the package."
from:
http://www.debian.org/doc/manuals/project-history/ch4.html
To change the subject slightly, there seem to be some other
dangerous related "purity before progress" idea bugs
floating around, like
garrigue@kurims.kyoto-u.ac.jp wrote:
> The real problem is about how to check that binary (and
> semantics) compatibility is satisfied.
Exactly what sort of semantics? Certainly the general case
is turing-complete. What reasonable subset could you mean
by this? And just why do we care? Nobody's pretending type
checking makes your program correct.
>>>>I don't see this. I can believe that consequences of
implementation
>>>>choices which have been made prohibit this. For
hypothetical example,
>>>>inlining behavior which could not be disabled on public
interfaces would
>>>>be a problem. (I don't think this particular thing
happens in ocaml.)
>>>>But I certainly don't see "a consequence of strict
typing."
>>>>
>>
>> I agree. It is not true of other languages with static t
yping,
>> e.g. Mercury or C++.
>
>I don't know Mercury but C++. Actually, it does not ensure
proper
>typing between compilation units! The .o files do not
contain types,
>and it is up to the programmer to include the right
prototypes.
Yes, it's called using the right header files. That and
version numbering your runtime libraries. Just like the
rest of the computing world. Apparently forced type
checking is an open problem, so lets not sit and cry over
it. Fortunately it hasn't been solved anywhere else, so it
is a universally understood programmer's convention to
couple libraries with source code interfaces. Are we really
so special that we must break it?
This is the scenario I envision. I don't think the design
choices above are hard, though there still may be some
substantial work because apparently a mutually exclusive set
of assumptions has governed the evolution of the compiler up
until this point.
- The compiler must guarentee a fixed calling convention
and representation for all concrete types for a given .mli
file.
- Types parameterized by abstract types (eg "list", sorry
if my jargon is ignorant) must not change their compilation
based on knowledge of a new type.
- Functors must have the analogous property
- If the above is not generally runtime efficient, then it
must be made so by providing a "public" or "external" or
some such keyword to allow the interfaces which are exposed
to have these properties.
(corrections/elaborations to the above are welcome feedback)
There _do_ currently exist design choices in the ocaml
sources that arbitrarily for no reason violate this. One of
them was fixed for SCaml, which was that arbitrary gensymed
numbers were being added to external function definitions
from ocamlopt. Eg I implement:
foo.ml
let f x = x + 1;;
and the compiler generates a .so symbol called "f17,"
whereas
foo.ml
let f x = x + x + 1;;
might generate "f23" or some other arbitrary thing. Merely
changing the length of procedure bodies can make a different
number of gensym requests happen and therefore the number 17
morphs into something else. Not good for linking against.
The SCaml patch does fix this particular thing, but there
are likely to be others.
I reiterate that I haven't seen a concrete counterexample
which demonstrates that the semantics themselves make
runtime interfaces impossible. A previous post of mine
answered with one counterexample which changed optional
arguments across a public interface. That won't work in C++
or any other of the myriad languages which do have
deterministic runtime interfaces. Apologies that I have not
read in detail a couple of examples using functors that came
in yesterday, but I had to finish this email sometime.
I additionally point out that on course granularity, dynamic
typing ala COM interface IDs and class IDs works pretty
well. (These are just large 128 bit numbers based on things
like the creator's ethernet hardware address and a fine
granularity time stamp, designed to be a unique type code
for every possible interface and class). The systems which
don't have the right library versions at least crash right
away when the module is linked rather than run for a while
and segfault when a function is utilized. It's not perfect,
but its ubiquity demonstrates that it works in practice.
In closing, I'd like to ask a more tenuous and subjective
question. Is there any fundamental reason why the caml
package manager can't _be_ the Debian package manager? I
think this would be a big step in the maturity and ubiquity
of the OCaml community at large. And I mean dependencies
_both_ ways, C calling Caml as well as Caml calling C and
other Caml. (Not that people these days are publishing a
lot of C interfaces for their Caml libraries. To integrate
with other programmer communities that should change but
it's another issue.) I haven't thought about this one a
lot, but having a quirky on-the-side package manager for a
special quirky language gives to me the general impression
of obscurity by design.
Jeff Henrikson
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
^ permalink raw reply [flat|nested] 26+ messages in thread* [Caml-list] Type-safe DLL's with OO (was DLL-hell of O'Caml)
2002-03-21 14:13 ` Jeff Henrikson
@ 2002-03-21 14:13 ` Tim Freeman
2002-03-21 18:10 ` [Caml-list] The DLL-hell of O'Caml Xavier Leroy
1 sibling, 0 replies; 26+ messages in thread
From: Tim Freeman @ 2002-03-21 14:13 UTC (permalink / raw)
To: jehenrik; +Cc: caml-list
From: "Jeff Henrikson" <jehenrik@yahoo.com>
>Imagine if the computer on your desk was built out of a (C
>based) source distro. Not just a source distro, but one
>with the "all downstream dependencies recompile" policy. (I
>don't think anybody has been so deluded to suggest this in a
>large system.) Now suppose you write device drivers.
>Device driver programmers were frustrated enough before the
>linux kernel was modularized so that they could load and
>unload them without relinking the kernel and rebooting. You
>mean to tell them now that they must recompile not only the
>kernel, but therefore the entire machine every time they
>make a one line bugfix?
No matter how this binary compatibility issue is resolved, you could
represent the API for the for the device driver as a virtual OCaml
object and have a function in the OS that registers a new instance of
a subclass of the virtual class as a device driver in future calls.
With this scheme, when you change a device driver, you wouldn't have
to recompile the world. You'd have to recompile the code that
registers the device driver, and everything from there to the top
level of the program. If the first thing the top level does is
register the new device driver, then that's just recompiling two
things, the device driver and the top level.
Unfortunately you'd then have to restart the "program" to get your new
device driver, and in this scenario the "program" is the operating
system. Yuck.
If we assume good object oriented design at the API for the device
driver, the minimal support you'd need to do this without restarting
the operating system is to be able to dynamically load a module and do
the static initialization for it. You don't need to access anything
inside the module. You don't need to get any information out of your
DLL except by side effects of the static initialization. In this case
the static initialization would do nothing but register new the device
driver.
I have sample code for this below. It deals with upgraded API's in a
type-safe way. It substitutes an "open" statement for loading and
initializing a DLL, since ocaml doesn't presently support loading
ocaml DLL's.
This would also solve the issue of the lablgtk executable size that I
raised earlier. The machinery is somewhat of a burden on the library
writer unless a code generator could be used.
I think this idea is mostly from Lakos' book "Large-Scale C++ Software
Design".
There would then be some value in teaching the garbage collector to
garbage collect object files from no-longer-used DLL's. Otherwise if
you load enough device drivers you run out of VM.
--
Tim Freeman
tim@fungible.com
P. S. Here's the illustration code. Version 1 of the library has a
function "boring". Version 2 of the library adds a function
"interesting" and a bugfix for "boring". There are three
applications: v1_user that uses version 1, v2_user that uses version
2, and v1_user_against_v2 that simulates the version 1 application
linked against the version 2 library. It sees the bugfix.
----- lib_static_v1_decl.ml -----
class virtual simple = object
method virtual boring: unit
end;;
----- lib_static_v1.ml -----
let v1_ptr:Lib_static_v1_decl.simple option ref = ref None;;
let boring () =
match !v1_ptr with
Some f -> f#boring
| None -> assert (false);;
----- lib_dynamic_v1.ml -----
class simpleImpl = object
inherit Lib_static_v1_decl.simple
method boring = print_string "Inside boring!\n"
end;;
Lib_static_v1.v1_ptr := Some new simpleImpl;;
----- v1_user.ml -----
(* I'd like to dynamically load Lib_dynamic_v1 here, but that*)
(* primitive doesn't exist, so I'll just open it instead. Note that I*)
(* never directly use anything from it. *)
open Lib_dynamic_v1;;
Lib_static_v1.boring ();;
----- lib_static_v2_decl.ml -----
class virtual fancy = object
inherit Lib_static_v1_decl.simple
method virtual interesting: unit
end;;
----- lib_static_v2.ml -----
let v2_ptr:Lib_static_v2_decl.fancy option ref = ref None;;
include Lib_static_v1;;
let interesting () =
match !v2_ptr with
Some f -> f#interesting
| None -> assert (false);;
----- lib_dynamic_v2.ml -----
class fancyImpl = object
inherit Lib_static_v2_decl.fancy
method boring = print_string "Inside bugfixed boring!\n"
method interesting = print_string "Inside interesting!\n"
end;;
Lib_static_v1.v1_ptr := Some (new fancyImpl :> Lib_static_v1_decl.simple);;
Lib_static_v2.v2_ptr := Some new fancyImpl;;
----- v2_user.ml -----
(* I'd like to dynamically load Lib_dynamic_v2 here, but that*)
(* primitive doesn't exist, so I'll just open it instead. Note that I*)
(* never directly use anything from it. *)
open Lib_dynamic_v2;;
Lib_static_v2.boring ();;
Lib_static_v2.interesting ();;
----- v1_user_against_v2.ml -----
(* The next line should really be loading a DLL. We should pick up a*)
(* different file because the old DLL was updated in place, so if we*)
(* could really load ocaml dll's, this file would be identical to*)
(* v1_user.ml. *)
open Lib_dynamic_v2;;
Lib_static_v1.boring ();;
----- Makefile -----
.DELETE_ON_ERROR:
EXES = v2_user v1_user v1_user_against_v2
.PHONY: run
run: $(EXES)
./v1_user
./v2_user
./v1_user_against_v2
.PHONY: clean
clean:
rm -f $(EXES) *.cmo *.cmi *~ make.dep
make.dep: $(wildcard *.ml)
ocamldep *.ml > make.dep
include make.dep
%.cmi %.cmo: %.ml
ocamlc -c $+
v1_user: lib_static_v1.cmo lib_dynamic_v1.cmo v1_user.cmo
ocamlc -o v1_user lib_static_v1_decl.cmo lib_static_v1.cmo lib_dynamic_v1.cmo v1_user.cmo
v2_user: lib_static_v2.cmo lib_dynamic_v2.cmo lib_static_v1.cmo lib_dynamic_v1.cmo v2_user.cmo
ocamlc -o v2_user lib_static_v1_decl.cmo lib_static_v2_decl.cmo lib_static_v1.cmo lib_dynamic_v1.cmo lib_static_v2.cmo lib_dynamic_v2.cmo v2_user.cmo
v1_user_against_v2: lib_static_v2.cmo lib_dynamic_v2.cmo lib_static_v1.cmo lib_dynamic_v1.cmo v1_user_against_v2.cmo
ocamlc -o v1_user_against_v2 lib_static_v1_decl.cmo lib_static_v2_decl.cmo lib_static_v1.cmo lib_dynamic_v1.cmo lib_static_v2.cmo lib_dynamic_v2.cmo v1_user_against_v2.cmo
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Caml-list] The DLL-hell of O'Caml
2002-03-21 14:13 ` Jeff Henrikson
2002-03-21 14:13 ` [Caml-list] Type-safe DLL's with OO (was DLL-hell of O'Caml) Tim Freeman
@ 2002-03-21 18:10 ` Xavier Leroy
2002-03-21 18:39 ` Sven
` (4 more replies)
1 sibling, 5 replies; 26+ messages in thread
From: Xavier Leroy @ 2002-03-21 18:10 UTC (permalink / raw)
To: Jeff Henrikson; +Cc: caml-list
> Ugh. I wholeheartedly disagree with source-only
> distribution. Such a choice condemns ocaml and all of its
> applications to remain merely "hacker tools". And all the
> worse to mandate a "all downstream dependencies recompile"
> policy. I for one need to be able to someday hope that
> "regular users" will use my program. Every ignorant user
> should _not_ be subjected to building and rebuild his own
> large system because of some deep semantics issue which
> researchers haven't quite cracked.
You're confusing whole programs with libraries. A program can be
distributed as a statically-linked executable (ocamlopt -ccopt -static
or ocamlc -custom -ccopt -static), which will run happily on many,
many systems, and users will never have to worry about binary
compatibility and whatnot. Only developers (= users of third-party
libraries) have to deal with code compatibility issues.
> Someone suggested the need of "closed source" as an argument
> for binary distribution. I can make an argument that relies
> on less. Assume ala mathematical induction OCaml
> proliferates. Foobar Corp does their bookkeeping in ocaml
> and has 2 million lines and needs to make a bugfix to their
> SQL wrapper way down at the bottom of the heap. Do you tell
> them they must wait a couple of days do get their system
> back up?
This is a strawman argument in many respects:
- The OCaml distribution is 100 KLOC and recompiles in a few minutes
on a 1000 EUR machine. So, 2MLOC takes less than one hour, not two days.
Chances are that if you're developing a program of this size,
you'll invest in a multiprocessor or compilation farm, bringing
the rebuild time even lower. (Say, just long enough to have coffee,
chat with your coworkers, and check Slashdot. Isn't programmer's life
sweet? :-)
- If I were to design an application of this size, I'll try very hard
not to make it into one executable, but rather decompose it into
several executables communicating via IPC. Hence, only the parts
that use the SQL wrapper have to be recompiled.
- Even if the program consists of one executable, it is easy to recompile
only the modules that really depend on the SQL library: instead of doing
"make clean; make all"; just generate dependencies with
ocamldep -I/the/path/to/the/SQL/library and do "make".
That we generally don't bother to record dependencies on
third-party libraries in Makefiles doesn't mean it can't be done
if need arises.
> Imagine if the computer on your desk was built out of a (C
> based) source distro. Not just a source distro, but one
> with the "all downstream dependencies recompile" policy.
This is not what I'm proposing. It is perfectly feasible to make a
binary OCaml "distribution" (in the sense of Debian and RedHat being
"distributions") such as the CDK. Someone (the distributor) just did
all the work of building a large collection of sources for you.
Indeed, a truly automated build procedure like I propose would make
the work of the CDK maintainers much easier.
What I think is futile is to expect users to pick independently-
compiled binary distributions of libraries and dynamically-linked executables
left and right, and expect that the resulting system will work. Try
to do this with a Linux system and it will break very quickly.
> (I don't think anybody has been so deluded to suggest this in a
> large system.) Now suppose you write device drivers.
> Device driver programmers were frustrated enough before the
> linux kernel was modularized so that they could load and
> unload them without relinking the kernel and rebooting. You
> mean to tell them now that they must recompile not only the
> kernel, but therefore the entire machine every time they
> make a one line bugfix? What kind of use would a machine
> like that be?
More rethorical questions that don't make much sense to me.
Loading/unloading kernel modules during kernel development is at best
a dynamic loading issue. That they might have to recompile their
driver when the kernel changes doesn't bother kernel developers in the
least.
> As a historical reference, the Debian project, having the
> hacker mentality they do, apparently also fanatasized about
^^^^^^^^^^^
I like this new word :-)
> having a source only distribution (of course I don't imagine
> it was an "all downstream dependencies recompile" design):
>
> >"At the very early stages of the Project, members
> considered
> >distributing source-only packages. Each package would
> consist
> >of the upstream source code and a Debianized patch file,
> and users
> >would untar the sources, apply the patches, and compile
> binaries
> >themselves. They soon realized, however, that some sort of
> >binary distribution scheme would be needed. The earliest
> packaging
> >tool, written by Ian Murdock and called dpkg, created a
> package in
> >a Debian-specific binary format, and could be used later to
> unpack
> >and install the files in the package."
Again, I don't have anything against binary distributions, as long as
there's an underlying source distribution and recompilation manager
that lets me bring things up to date if a binary incompatibility
arises. RPMs get close to this but lack a "rebuild and install what
needs to be recompiled" command. Maybe Debian's packages or BSD ports
are better in this respect.
> To change the subject slightly, there seem to be some other
> dangerous related "purity before progress" idea bugs
> floating around, like
And there seems to be some dangerous "let's hack something before
thinking over the consequences" ideas in your message.
- Xavier Leroy
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Caml-list] The DLL-hell of O'Caml
2002-03-21 18:10 ` [Caml-list] The DLL-hell of O'Caml Xavier Leroy
@ 2002-03-21 18:39 ` Sven
2002-03-21 19:22 ` james woodyatt
` (3 subsequent siblings)
4 siblings, 0 replies; 26+ messages in thread
From: Sven @ 2002-03-21 18:39 UTC (permalink / raw)
To: Xavier Leroy; +Cc: Jeff Henrikson, caml-list
On Thu, Mar 21, 2002 at 07:10:28PM +0100, Xavier Leroy wrote:
> > Ugh. I wholeheartedly disagree with source-only
> > distribution. Such a choice condemns ocaml and all of its
> > applications to remain merely "hacker tools". And all the
> > worse to mandate a "all downstream dependencies recompile"
> > policy. I for one need to be able to someday hope that
> > "regular users" will use my program. Every ignorant user
> > should _not_ be subjected to building and rebuild his own
> > large system because of some deep semantics issue which
> > researchers haven't quite cracked.
>
> You're confusing whole programs with libraries. A program can be
> distributed as a statically-linked executable (ocamlopt -ccopt -static
> or ocamlc -custom -ccopt -static), which will run happily on many,
Unless they are not running i386 that is, ...
>
> This is not what I'm proposing. It is perfectly feasible to make a
> binary OCaml "distribution" (in the sense of Debian and RedHat being
> "distributions") such as the CDK. Someone (the distributor) just did
> all the work of building a large collection of sources for you.
>
> Indeed, a truly automated build procedure like I propose would make
> the work of the CDK maintainers much easier.
>
> What I think is futile is to expect users to pick independently-
> compiled binary distributions of libraries and dynamically-linked executables
> left and right, and expect that the resulting system will work. Try
> to do this with a Linux system and it will break very quickly.
>
> > (I don't think anybody has been so deluded to suggest this in a
> > large system.) Now suppose you write device drivers.
> > Device driver programmers were frustrated enough before the
> > linux kernel was modularized so that they could load and
> > unload them without relinking the kernel and rebooting. You
> > mean to tell them now that they must recompile not only the
> > kernel, but therefore the entire machine every time they
> > make a one line bugfix? What kind of use would a machine
> > like that be?
>
> More rethorical questions that don't make much sense to me.
> Loading/unloading kernel modules during kernel development is at best
> a dynamic loading issue. That they might have to recompile their
> driver when the kernel changes doesn't bother kernel developers in the
> least.
>
> > As a historical reference, the Debian project, having the
> > hacker mentality they do, apparently also fanatasized about
> ^^^^^^^^^^^
This must have been a long time ago, today there is no quation on going away
with biunary package, altough there still exist some exception (like pine i
think, where only source distribution is authorized).
But then again, we distribute various rather slow arches (m68k and arm), and
there may be user on slow i386 systems out there also, i don't think they
would like to rebuild everything everytime something changed.
Binary distribution are the key of the success of GNU/Linux, without it, it
never would have attained the widespread use it has today.
> Again, I don't have anything against binary distributions, as long as
> there's an underlying source distribution and recompilation manager
> that lets me bring things up to date if a binary incompatibility
> arises. RPMs get close to this but lack a "rebuild and install what
> needs to be recompiled" command. Maybe Debian's packages or BSD ports
> are better in this respect.
on debian you can just do :
apt-get source -b ocaml
and it will fetch the ocaml source package and build it, you still need to
install it yourself.
There are build dependencies in package, it is possible to check for them, to
have them automatically installed when building a package, and other such
stuff, but i am not very familiar with them, it being a long time since i did
porting work (mostly m68k and powerpc).
We also have autobuilders, which build stuff for all the majors arches.
Friendly,
Sven Luther
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Caml-list] The DLL-hell of O'Caml
2002-03-21 18:10 ` [Caml-list] The DLL-hell of O'Caml Xavier Leroy
2002-03-21 18:39 ` Sven
@ 2002-03-21 19:22 ` james woodyatt
2002-03-21 19:43 ` Jeff Henrikson
` (2 subsequent siblings)
4 siblings, 0 replies; 26+ messages in thread
From: james woodyatt @ 2002-03-21 19:22 UTC (permalink / raw)
To: Xavier Leroy; +Cc: Jeff Henrikson, caml-list
On Thursday, March 21, 2002, at 10:10 AM, Xavier Leroy wrote:
>
> [...] You're confusing whole programs with libraries. [...] Only
> developers (= users of third-party libraries) have to deal with code
> compatibility issues.
I'm a practitioner and not a researcher, so I share many of Mr.
Henrikson's concerns, but I think I see your point.
Most of my concerns revolve around the problems I expect to face in
bundling an application that comprises many little programs that share
one or more large libraries. I'd like to see better support for dynamic
loading, á la Scaml, but I can see that a policy of "source-only
distribution" for libraries between developers is a completely different
matter.
While the "source-only distribution" policy doesn't put a clamp on my
personal plans, I do think that this decision will have to be revisited
before Ocaml will make a successful transition from academia into
industry.
One of the main attractions for industry to the C and Java languages are
their crude-yet-functional support for "closed-source distribution" of
libraries. Without trying to make a political statement, I would
suggest that Ocaml could make a *much* bigger dent in the side of the
world, if the Caml team were to reconsider the concerns of industry in
this matter.
I've already posted my opinion about application packaging systems, but
it's not a religious issue for me, so I'll just let it go.
--
j h woodyatt <jhw@wetware.com>
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
^ permalink raw reply [flat|nested] 26+ messages in thread* RE: [Caml-list] The DLL-hell of O'Caml
2002-03-21 18:10 ` [Caml-list] The DLL-hell of O'Caml Xavier Leroy
2002-03-21 18:39 ` Sven
2002-03-21 19:22 ` james woodyatt
@ 2002-03-21 19:43 ` Jeff Henrikson
2002-03-22 2:02 ` Brian Rogoff
2002-03-22 10:11 ` Warp
4 siblings, 0 replies; 26+ messages in thread
From: Jeff Henrikson @ 2002-03-21 19:43 UTC (permalink / raw)
To: Xavier Leroy; +Cc: caml-list
> You're confusing whole programs with libraries.
> A program can be
> distributed as a statically-linked executable . . .
Ideally, I'd like to confuse them as much as possible. From
the libtool manual: "libraries are programs with multiple
entry points, and
more formally defined interfaces."
I think a good rule of thumb for a rational developer
prospectively looking at using a new language is "How easy
is <insert here> compared to C?" Insert your favorite term
[package management/system integration/dynamic library
futzing]. I claim our choice in package management should
make sense to a good but non-ocaml-indoctrinated programmer.
Right now I don't think the proposals fall along those
lines.
Ideally, multiple ocaml libraries loaded into the same
process could do all the same things as C:
- share (ocaml) runtimes
- call each other
- be loaded and unloaded
- call and be called by other languages without much
programmer effort
and at build time:
- if no interfaces are changed rebuild only one shared lib
and rebuild only that. (In fact newer Linux can even reload
such a recompiled lib while the process is still running,
hence you can upgrade libc while still running, etc.)
Currently, certain tools to make this so are missing. (For
example both forward and reverse stub generators smarter
than camlidl and camoflage.) But we should not make package
management design choices that preclude these possibilities
unless there's a good reason to.
> This is a strawman argument in many respects:
>
> - The OCaml distribution is 100 KLOC and recompiles in a
few minutes
> on a 1000 EUR machine. So, 2MLOC takes less than one
hour, not two
> days. . . .
I am admittedly offering arbitrary unchecked numbers. But I
don't think the argument changes, it's analagous to an order
of growth argument. If build times don't matter, then why
does "make" exist? I for one am quite impatient to build
caml on MPW with a broken make tool (or makefiles? I haven't
quite taken the time to understand.) which recompile the
world upon one file touch. I heard Gerry Sussman (scheme
coinventor) quip something once along the lines of "Software
is like some funny kind of gas. It takes up all the space
you give it." I think this applies to build times as well
as run times.
> > To change the subject slightly, there seem to
> be some other
> > dangerous related "purity before progress" idea bugs
> > floating around, like
>
> And there seems to be some dangerous "let's hack
> something before
> thinking over the consequences" ideas in your message.
Admittedly. I would probably think differently if I
everything were up to me, but since I know there are enough
people of the opposite thinking to keep me in check, I feel
it's most useful for me to err on this side. In groups with
abundance of hacker mentality I am quite happy to play the
other role. Sorry if anyone is offended.
Regards,
Jeff Henrikson
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Caml-list] The DLL-hell of O'Caml
2002-03-21 18:10 ` [Caml-list] The DLL-hell of O'Caml Xavier Leroy
` (2 preceding siblings ...)
2002-03-21 19:43 ` Jeff Henrikson
@ 2002-03-22 2:02 ` Brian Rogoff
2002-03-22 10:11 ` Warp
4 siblings, 0 replies; 26+ messages in thread
From: Brian Rogoff @ 2002-03-22 2:02 UTC (permalink / raw)
To: Xavier Leroy; +Cc: Jeff Henrikson, caml-list
On Thu, 21 Mar 2002, Xavier Leroy wrote:
> > Someone suggested the need of "closed source" as an argument
> > for binary distribution. I can make an argument that relies
> > on less. Assume ala mathematical induction OCaml
> > proliferates. Foobar Corp does their bookkeeping in ocaml
> > and has 2 million lines and needs to make a bugfix to their
> > SQL wrapper way down at the bottom of the heap. Do you tell
> > them they must wait a couple of days do get their system
> > back up?
>
> This is a strawman argument in many respects:
In fact, lot's of commercial entities do provide upgrades in one swell
foop, rather than providing lots of little patches. I wasn't sure about
the history, but I asked the guy who supports all of our CAD software this
morning and he wrote
EDA companies used to provide software patches
which were a few files. These days, most of
them provide monolithic releases which are
generally self-sufficient, and incorporate multiple
bug fixes, in order to minimize the number of
custom versions they have to support.
I actually expected the use of shared libraries to be increasing over
time. Oh well, that just shows you how closely expectations and reality
conform.
Anyways, this isn't an issue that's ever bothered any of the OCaml fans
where I work. Jeff, if you want to cause trouble, suggest that optimized
code and bytecode should work together seamlessly in all combinations.
That's very desirable, very hard to do, and there are no theory problems
there, just lots of grunge work :-).
-- Brian
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Caml-list] The DLL-hell of O'Caml
2002-03-21 18:10 ` [Caml-list] The DLL-hell of O'Caml Xavier Leroy
` (3 preceding siblings ...)
2002-03-22 2:02 ` Brian Rogoff
@ 2002-03-22 10:11 ` Warp
4 siblings, 0 replies; 26+ messages in thread
From: Warp @ 2002-03-22 10:11 UTC (permalink / raw)
To: OCaml
> Xavier Leroy wrote :
>
> Again, I don't have anything against binary distributions, as long as
> there's an underlying source distribution and recompilation manager
> that lets me bring things up to date if a binary incompatibility
> arises. RPMs get close to this but lack a "rebuild and install what
> needs to be recompiled" command. Maybe Debian's packages or BSD ports
> are better in this respect.
Are you expecting all the OCaml developpers to distribute full source code
with each of the binary distribution they're making ? That's perhaps
realistic from a Linux/FSF point of view, but not from a Windows one.... To
enable the widespread of such a good language, the dev community needs solid
and useful libraries to build up a complete dev environment - including DB
access, Network , mathematics, drawing, etc... This work has already start
but some won't release their sources because they're in businness or because
sources are not "clean" enough and releasing them in a proper way would cost
a lot of time (and money)... or many many many reasons which have made lots
of commonly C/C++ used libraries been closed source.
I have worked a lot with Dynlink, and currently the main lack is to enable
"dynamic-link at load-time". That is, adding a keyword ( you're writing for
example "external" instead of "open" ) and then ocamlrun will load the CMA
at startup, check the interface, and do the linking.
If you combine such a feature with the ability to check only the equality
between the sub-interface of the module your program is using , then you're
not breaking type-safe at all, and you allow binary-only-distribution.
With a little bit of versionning, that'll be perfect :)
Nicolas Cannasse
the only window ocaml user (?) :-)
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Caml-list] The DLL-hell of O'Caml
2002-03-20 20:39 ` Xavier Leroy
` (3 preceding siblings ...)
2002-03-21 14:13 ` Jeff Henrikson
@ 2002-03-21 18:50 ` Sven
4 siblings, 0 replies; 26+ messages in thread
From: Sven @ 2002-03-21 18:50 UTC (permalink / raw)
To: Xavier Leroy; +Cc: Gerd Stolpmann, Johan Georg Granström, caml-list
On Wed, Mar 20, 2002 at 09:39:25PM +0100, Xavier Leroy wrote:
> But A.cmo does not contain SIGB -- only a mention of B and a hash of
> SIGB. The reason is quite simple: A.cmo would be *huge* if it
> included a copy of the signatures of every module it refers, either
> directly or indirectly. (Think several megabytes.)
Just curious, ...
would it not be enough if A.cmo had the type information of the part of B it
is going to use, and forget about the rest of it ? I remember some papers on
type discipline doing this kind of stuff.
> Having only hashes of signatures, all the linker can check is
> hash(SIGB) = hash(SIGB'), that is SIGB = SIGB'.
>
> > Anyway, only being able to add functions is not sufficient. There are
> > many possible changes of modules that would not break "source-code
> > compatibility", but would certainly break binary compatibility, e.g.
> > new optional arguments, new variants, new methods, etc.
>
> This is another good point: even if we allowed SIGB' <: SIGB at
> link-time instead of requiring SIGB = SIGB', there are many
> "reasonable" changes in a signature that would break subtyping, like
> those you mention above. And this is inevitable: these changes could
> break type-safety of existing programs.
>
> To summarize: I believe binary compatibility doesn't really work in a
> strongly, statically-typed language. It sometimes works for
> dynamically-typed or weakly-typed languages (Perl, C), although
> run-time failures are not unheard of. Copious link-time
> re-type-checking as in Java makes it work sometimes too, although on
> close examination Java's binary compatibility rules are quite complex
> and restrictive.
>
> Conclusion for OCaml? Forget about binary compatibility, binary-only
> distributions, versioning, and all that. All we need for OCaml
> libraries is source distribution, a standardized re-building procedure,
> and a place to record cross-library dependencies, so that everything
> that needs to be recompiled when something changes can be recompiled
> automatically. Something like a source RPM or Debian package, or a
> BSD port.
Even for that, it is nice to have propper versioning support, right now, the
policy for debian packagers, is once the ocaml package changes (new upstream
release,, which more often than not breaks binary compatibility, but sometimes
only partially)., then all packagers need to recompile all the ocaml
libraries, and just to be sure, it is best to rebuild the program packages
also. But this needs still human intervention on the part of the developpers
to adapt the package if needed, and a lot of work for all the 8 arches
which will be supported by woody, without even counting all those not yet
fully supported arches which don't have autobuilders.
I would like a lot to have more information on versioning available.
> I believe this is a much more reasonable objective than dreaming about
> binary compatibility and whatnot. Still, it's not trivial:
> the authors of existing OCaml libraries (myself included) have some
> work to do to implement fully automatic build of their libraries...
:)))
Friendly,
Sven Luther
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
^ permalink raw reply [flat|nested] 26+ messages in thread